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Introduction 

Mathematical billiards is a rich and beautiful subject. It is very extensive as well. The choice of 
material for this survey reflects the taste of the author, who has attempted to make the exposition 
as geometrical as possible. 

The survey consists of five chapters: the first provides some general background; the second 
concerns convex smooth billiards (elliptic case); the third deals with billiards in polygons and 
polyhedra (parabolic case); the fourth discusses the lesser-known topic of dual billiards, which 
are of particular interest to the author; and the fifth is a very brief treatment of chaotic billiards 
(hyperbolic case). Each chapter has a brief introduction of its own and is subdivided into sections; 
it goes without saying that "Lemma 1.2.3" means "Lemma 3 from Section 2, Chapter 1". 

The author is grateful to the mathematicians he had the opportunity to discuss billiards with 
and learn from: V. Arnold, M. Audin, M. Berger, M. Bialy, Ph. Boyland, N. Chernov, D. Fuchs, 
G. Galperin, E. Ghys, A. Givental, M. Gromov, E. Gutkin, P. Iglesias, A. Katok, R. de Llave, J. 
Moser, Ya. Pesin, L. Polterovich, Ya. Sinai, J. Smillie, S. Troubetzkoy, A. Veselov, M. Wojtkowski. 
They taught him far more than these notes show. 

Special gratitude goes to I. Monroe for his help with the numerical study of dual billiards, to 
J. Duncan for patiently reading the text and giving stylistic advice and to E. Gutkin, G. Galperin 
and the referees whose suggestions helped to improved the exposition. Last but not least, it is a 
pleasure to acknowledge the partial support of an ASTA grant 94-B-25. 
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1. General Theory and Mathematical Background 



This chapter concerns the definition of the billiard transformation and the billiard flow. It 
also provides a necessary mathematical background for the rest of the survey. Section 2 introduces 
the area form invariant under the billiard transformation in the two-dimensional case, and the 
variational approach to the billiard problem. Section 3, in the spirit of the geometrical optics, 
deals with the billiard transformation of the space of rays in the plane. Section 5 is a very 
brief introduction to symplectic geometry, which we use in the next section to understand the 
results of Sections 2 and 3 from a more general viewpoint and to generalize them to the higher- 
dimensional case. Section 7 concerns discontinuities of the billiard transformation; we show that 
the transformation and the billiard flow for all times are defined almost everywhere in the sense of 
measure. Section 10 introduces the hierarchy of stochastic properties such as ergodicity, minimality, 
topological transitivity, mixing, etc. The contents of other section is self-explanatory. 
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1.1 Mathematical Billiards. Phase and Configuration Spaces 

A billiard table is a Riemannian manifold M with a piecewise smooth boundary. The billiard 
dynamical system in M is generated by the free motion of a mass-point (called a billiard ball) 
subject to the elastic reflection in the boundary. This means that the point moves along a geodesic 
line in M with a constant (say, unit) speed until it hits the boundary. At a smooth boundary 
point the billiard ball reflects so that the tangential component of its velocity remains the same, 
while the normal component changes its sign. In dimension two this collision is described by a 
well known law of geometrical optics: the angle of incidence equals the angle of reflection. Thus 
the theory of billiards and the theory of geometrical optics have many features in common. If the 
billiard ball hits a corner, its further motion is not defined (there are some exceptions to this to 
be discussed later). 

fig. 1 

The time-i billiard transformation acts on unit tangent vectors to M, more precisely, on those 
pairs (x, v) with x £ M, v £ T X M whose trajectories undergo finitely many reflections in the 
boundary and avoid corners on the time interval [0,t]. The unit tangent bundle to M is the phase 
space of the billiard, and the manifold M is its configuration space. 

Billiards are the geodesic flows on Riemannian manifolds with boundaries. They can also 
be treated as a limit case of the geodesic flows on boundary less manifolds, at least heuristically. 
Let M be a smooth plane billiard table. Consider its "thickening", i.e. an infinitely thin three 
dimensional body whose boundary N is obtained by pasting two copies of M along their boundaries 
and smoothening the edge. Then a billiard trajectory in M can be viewed as a geodesic line on 
the boundary of N, that goes from one copy of M to another each time the billiard ball bounces 
off the boundary. This construction is due to G. Birkhoff ([Bi 1]). 

fig. 2 

1.2 Invariant Measure and Generating Function for Plane Billiards. Vari- 
ational Formulation 

So far billiards were defined as a continuous time dynamical system. One can reduce the 
dimension by one and replace continuous time by discrete time, i.e. replace a flow by a mapping. 

Let M be a bounded plane billiard table. Consider the manifold V of unit tangent vectors 
(x, v) with the inward direction v and the footpoint x on the boundary dM. If the boundary 
consists of one component then V is an annulus S 1 x /. We now define the billiard transformation 
T of V. A vector (x, v) moves along the straight line through x in the direction of v to the next point 
of its intersection x\ with dM, and then v reflects in dM to a new vector v\: T(x,v) = (x\,Vi). 
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fig. 3 

A very remarkable property of the billiard transformation T is the existence of an invariant 
area form. Parametrize dM by the length parameter t and let a be the angle between v and the 
direction of the boundary at the point x(t). Use (t, a) as the coordinates in V, a 6 [0,7r]. 

Lemma. The area form sin a da A dt is T-invariant. 

Proof. Let T(t, a) = (t 1} ai). One wants to prove that sino?! da x A dt 1 = sin a da A dt. 
Let H(t,t\) be the distance between the points x(t) and x(t\). It readily follows from elementary 
geometry that 

dH(t,h) , dH(t,h) 

= — cos a and = cos a\ . 

at at i 

Hence 

dH(t,h) , dH(t,t x ) , 

cos Q!i ciii — cos a dt = dt\ H dt = dH, 

at i at 

and taking differentials, 

sin ai da\ A dt\ — sin a da A dt = 0. 

Q.E'.D. 

Consider three consecutive points: (t\,a{) = T(t,a),(t 2 ,a2) = T(ti,ai). It follows from the 
previous proof that 

dH(t,h) dH(t!,h) 



cos a\ , = — cos a\ . 



Hence 



dh " dt! 

dH(t,h) | dH{h,t 2 ) =Q 



dt! Oh 

This formula has the following interpretation. Suppose that one wants to start the billiard 
ball at the point x so that after one reflection in the boundary at some point xi it arrives to the 
given point x 2 - How does one find the unknown point xi? Answer: this point is a critical point of 
the functional dist(x,x{) + dist(x!, x 2 ). This variational principle plays an important role in the 
study of billiards. 

1.3 Billiard Transformation of the Space of Rays in the Plane 

It would be more in the spirit of geometrical optics to deal with oriented lines (or rays). Such 
an approach to billiards is possible and indeed fruitful. 

Suppose that a plane billiard table M is convex (this assumption is not really necessary; what 
follows will hold true locally for a generic M). Let U be the set of oriented lines in the plane 
that intersect M. To parametrize the set of rays choose the origin O inside M. Given a ray I, 
drop the perpendicular OP onto it. Fix a direction in the plane and let I make the angle of 4> 
with it. Let p = ±\OP\ depending on the orientation of the frame (I, OP). Then (p,<j>) are the 
coordinates in the set of oriented lines in the plane. U is given by the inequality \p\ < f(4>), where 
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the function f(eft) depends on the shape of M (it is called the support function). It follows that U 
is diffeomorphic to an annulus. 

fig. 4 

Define the billiard transformation T 1 oiU: the ray, that contains a segment of the trajectory 
of the billiard ball, oriented by the direction of its motion, is sent to the ray, that contains the next 
segment of this trajectory after the reflection in the boundary. 

The set of lines in the plane is an object of study in integral geometry. It is known that there 
exists a unique, up to a constant factor, measure on the set of lines invariant under the motions of 
the plane. In our notation this is given by the 2-form dp A deft (see [Sa], and also [Ig] on the relation 
between symplectic and integral geometry) . 

Identify the manifold V, introduced in the previous section, with U: the ray through x in the 
direction of v corresponds to a point (x, v) 6 V. Thus one identifies the transformations T and T' . 
Compare the two 2-forms sin a da A dt and dp A d(j>. 

Lemma. These forms are equal. 

Proof. An equation of the line, whose coordinates in U are (p, eft), is 

y cos eft — x sin eft = p. 

Differentiate: 

cos eft dy — sin eft dx — (y sin + x cos eft) d(j> = dp. 

Hence 

cos (ft dy A d(f) — sin (ft dx A deft = dp A deft. 

The angle made by the direction of dM at the point x(t),y(t) and the fixed direction is a + (ft. 
Therefore 

dy = sin(a + (ft) dt, dx = cos(a + (ft) dt. 



fig- 5 
Hence 

(cos0sin(a + (ft) — sin</>cos(a + (ft)) dt A deft = sin a dt A deft = dp A deft. 

Since ^"^^ = K{t) -the curvature of the curve dM, one has: deft = —da + K dt. Therefore 
dt A deft = da A dt and, finally, 

sin a da A dt = dp A deft 

Q.E.D. 
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It follows that the billiard transformation X" preserves the natural area form of the space of 
rays in the plane. 

1.4 Language of Symplectic Geometry 

To put the observations, made so far about plane billiards, into a proper context, and to 
generalize them to higher dimensions, one needs some basic concepts of symplectic geometry (see 
the excellent surveys [A-G, Ar 1, Gro]). 

Definition. A symplectic manifold (M, to) is a smooth manifold M with a closed nondegen- 
erate differential 2-form u>, called the symplectic structure. 

Since the 2-form is nondegenerate, the dimension of a symplectic manifold is even. A sym- 
plectic manifold has a canonical volume form uj n , where 2n = dim M. 

Example. A 2n-dimensional linear space with coordinates xi, x n , y±, y n has a linear 
symplectic structure dx A dy = dx\ A dy\ + ... + dx n A dy n . 

Unlike Riemannian manifolds symplectic manifolds are all locally equivalent (Darboux's the- 
orem): there exists a local diffeomorphism that carries one symplectic form to another. Hence the 
previous example provides a local normal form of a symplectic manifold. The coordinates x, y in 
which uj = dx Ady are called Darboux coordinates. 

The following example is of fundamental importance to classical mechanics, and in particular, 
to the theory of billiards. 

Example. The cotangent bundle T*M of a smooth manifold M has a symplectic structure. 
Let A be the tautological differential 1-form on T*M (called the Liouville form), i.e., the form 
whose value on a vector £, tangent to T*M at a point (q,p) with q G M, p 6 T*M, is equal to the 
value of the covector p on the projection of £ to the tangent space T q M. The natural symplectic 
structure on T*M is the 2-form u> = dX. 

Identify a linear 2n-dimensional space with the cotangent bundle of an n-dimensional space, 
and choose the "position" coordinates qi,...,q n and the dual "momentum" coordinates pi, ...,p n - 
In these coordinates A = p dq = p\ dq\ + ... +p n dq n and uo = dp Adq = dpi A dq\ + ... + dp n A dq n . 

Definition. An n-dimensional submanifold L of a symplectic manifold (M 2n ,u>) is called 
Lagrangian if the restriction of u to L vanishes. 

Since u is a nondegenerate 2-form, n is the greatest possible dimension of a submanifold on 
which the symplectic form vanishes. 

Examples. A smooth curve in the plane (with any area form) is a Lagrangian manifold. 
Fibers of a cotangent bundle are Lagrangian manifolds. Given a smooth function on a manifold 
M the graph of its differential, considered as a section of the cotangent bundle, is a Lagrangian 
submanifold in T*M. 

Definition. A diffeomorphism of symplectic manifolds that carries one symplectic structure 
to another is called a symplectomorphism. 

Symplectomorphisms carry Lagrangian manifolds to Lagrangian manifolds. If /: (Mi,o»i) — > 
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(M 2 , 0J2) is a symplectomorphism then its graph is a Lagrangian submanifold of the product man- 
ifold Mi x M 2 with the symplectic structure u>i 0w 2 . The converse is also true. 

Definition. A symplectic manifold is called exact if its symplectic structure is the differential 
of a 1-form: to = dX. 

Cotangent bundles are exact symplectic manifolds. An exact symplectic manifold cannot be 
compact since its symplectic volume form is exact. Let T be a symplectomorphism of an exact 
symplectic manifold. Then T*co = to, or d (T*X — A) = 0. 

Definition. A symplectomorphism is called exact if the closed 1-form T* X — A is exact: 
T*X — A = dH for a function H. The function H is called a generating function of the exact 
symplectomorphism. 

To a function on a symplectic manifold a vector field naturally corresponds (the function is 
sometimes called a Hamiltonian function and the field - the Hamiltonian vector field). A symplectic 
structure, being a nondegenerate 2-form, defines an isomorphism between the tangent and the 
cotangent bundles. The Hamiltonian vector field (a section of the tangent bundle) corresponds 
under this isomorphism to the differential of the Hamiltonian function (a section of the cotangent 
bundle). In Darboux coordinates the Hamiltonian field of a function / is — §^^: + f^^- The 
flow of a Hamiltonian vector field preserves the symplectic structure. In particular, it preserves 
the symplectic volume (Liouville's theorem). The Hamiltonian vector field of a function / is also 
called its symplectic gradient and denoted by sgrad f. 

Hamiltonian vector fields form a Lie algebra under the usual commutator of vector fields. 
Hamiltonian functions also form a Lie algebra whose operation is called Poisson bracket. The 
Poisson bracket of two functions / and g is the derivative of one of them along the Hamiltonian 
vector field of the other. In Darboux coordinates {/, g} = — f^ff • The map / — ► sgrad f 

is a homomorphism of Lie algebras. 

Given a hypersurface in a symplectic manifold, the restriction of the symplectic structure to 
it is not nondegenerate any more: it has a one-dimensional kernel in each tangent hyperplane. 

Definition. This kernel is called the characteristic direction. Integral lines of the field of 
characteristic directions are called characteristic lines, or simply characteristics. Characteristic 
lines constitute the characteristic foliation of a hypersurface. 

If a smooth function is constant on a hypersurface then its symplectic gradient is tangent to 
the hypersurface along its characteristic lines. 

Suppose that the set of characteristics of a hypersurface in a symplectic manifold is itself 
a manifold (locally it is always the case). Then this new manifold of characteristics carries a 
symplectic structure: its value at a pair of tangent vectors £ and fj equals the value of the original 
symplectic structure at vectors £ and 77, tangent to the hypersurface at some point, that project 
to £ and 7) (the result does not depend on the choice involved). 

Let M be a Riemannian manifold. Consider the hypersurface in T*M that consists of unit 
covectors. The characteristics of this hypersurface are identified with nonparametrized oriented 
geodesic lines in M. Hence if the set of oriented geodesies of M is a manifold, it is a symplectic 
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manifold. 

Example. The set of oriented lines in a Euclidean space is a symplectic manifold. Up to the 
sign of the symplectic structure it is symplectomorphic to the cotangent bundle of the unit sphere 
in the space. The diffeomorphism of the space of rays with T*S n ~ l is shown in the figure (exercise 
for the reader: formulate it explicitly). 

fig. 6 

1.5 Symplectic Properties of Billiards 

Now we are in a position to reconsider the results of Sections 1.2—1.3 from a broader point of 
view. To fix ideas, consider a bounded strictly convex domain M with a smooth boundary in an 
re-dimensional Euclidean space. This will be the billiard table (in the general case the results of this 
section hold locally, that is, in a neighbourhood of a generic point of the phase space). Identify the 
tangent and cotangent bundles using the Euclidean structure, and consider the natural symplectic 
structure of T*M. Two hypersurfaces in T*M are of importance: the one that consists of unit 
(co)vectors, and the one that consists of (co)vectors whose footpoints lie on the boundary dM. 
Call these hypersurfaces Y and Z, correspondingly. Their intersection W consists of unit tangent 
vectors with the footpoints on dM. This intersection is transversal. 

Consider the characteristic foliations of the hypersurfaces Y and Z. We already know that 
the characteristics of Y are oriented lines in the space, that intersect M. Let U be the manifold 
of these lines; it has the induced symplectic structure, introduced in the previous section. U is 
diffeomorphic to the unit disc subbundle of the cotangent bundle of the unit sphere in the space. 

To describe the characteristics of Z, consider the map Z — > T{dM) that projects a tangent 
vector in the ambient space onto the tangent hyperplane to dM. 

Lemma 1. The characteristics of Z are the hbers of this projection. 

Proof. Introduce the usual coordinates q\, q n ,pi, —,p n in the cotangent bundle of the 
space; oj = dp A dq. Let f(q) = be a local equation of dM. Then the fibers of the projection 
Z — > T(dM) are generated by the normal vectors to dM, i.e., by the vectors d ^g^ -§- ■ The inner 
product of this vector with uj equals df - a 1-form that vanishes on tangent hyperplanes to dM. 
Q.E.D. 

Thus the space of characteristics of Z is the (co)tangent bundle of the boundary of the billiard 
table. Call it V. The identification V = T*{dM) carries the symplectic structure of V to that of 
T*(dM). The image of the composite map W — > Z — > V is the unit disc subbundle of T(dM). 
This image can be identified with the set of inward unit tangent vectors to M whose footpoints 
lie on dM. In the two-dimensional case the symplectic structure of V is given by the formula 
sin a da A dt from Section 1.3. 

Let S C W be the set of points where the restriction of the symplectic structure of T*M to W 
degenerates. Denote T*M by X and collect all the spaces and maps in the commutative diagram: 
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X 

Y Z 

w 

U V 

s 

Here (X, co) is a symplectic manifold; Y and Z are two hypersurfaces, whose transversal 
intersection is W; U and V are the spaces of characteristics of Y and Z; and X is the singular 
set of the restriction of u> onto W. This hexagonal diagram was introduced by R. Melrose ([Me 
1,2; Ar 2,3]) in his study of diffraction singularities near gliding rays. This diagram translates 
the differential geometry of submanifolds in a Riemannian manifold to the symplectic geometry of 
pairs of hypersurfaces in a symplectic manifold (not necessarily in the context of billiards). 

Lemma 2. £ equals the set of critical points of the projection W — > U as well as of the projection 

Proof. A point of W is a critical point of the projection W — > U if and only if the characteristic 
of Y trough this point is tangent to W. In this case the restriction of cu to W has a kernel - this 
characteristic direction. Conversely, let £ be a vector from the kernel of the restriction of u> to W, 
and let r\ be a characteristic vector of Y at the same point. If £ and ry are not collinear, then the 
tangent space to Y is generated by r] and the tangent space to W. Since r?) = 0, £ also belongs 
to feer u;|y. Hence £ also has the characteristic direction and is collinear to r\. Thus n is tangent 
to W. Q.E.D. 

In the case of billiards S consists of unit (co)tangent vectors to the boundary <9M. Convexity 
of the billiard table implies that the only singularity of the projection W — ► 27 is a fold along S 
(and similarly for — > V). Two involutions on arise: the ones that interchange the inverse 
images of a point under the projections W — > U and W — > V. Call them a and r, correspondingly. 
The two involutions have the common set of fixed points S. The billiard transformation of W is 
the composition to. 

fig. 7 

The results of Sections 1.2-1.3 in our more general situation will follow from an almost tauto- 
logical remark. 

Lemma 3. The projections W — > U and W — > F preserve the symplectic structures off the 
singular set S. The involutions a and r preserve the restriction of uj to W. 

Proof. Given two tangent vectors to U, one computes the value of the symplectic structure 
of U on these vectors as follows: lift the vectors to W and evaluate u on them. This implies the 
first statement. 

Given two tangent vectors to W, one computes the value of u\W on them as follows: lift the 
vectors to Y and evaluate oj\Y on them. Since the result does not depend on the lift, one could 
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equally well use the images of these tangent vectors to W under the involution a. This implies the 
second statement. Q.E.D. 

Finally one defines the billiard transformations T' : U — > U and T : V — > V. Define T' . 
Given a ray in the space, that intersects M (i.e., a point in U), consider its unit tangent vector at 
the second point of its intersection with dM (lift the point of U to W). Apply r (reflect in the 
boundary) and project back to U. Likewise one defines T. Given a tangent vector to dM, whose 
length is not greater than 1 (a point in V), consider the inward unit vector in the ambient space at 
the same point of dM whose projection to T(dM) is the given vector (lift to W). Apply a (move 
the billiard ball until it hits the boundary) and project to V. 

It follows from the previous lemma that the transformations T and T' preserve the symplectic 
structures of V and U. Therefore they also preserve the symplectic volumes therein. We also 
mention that T and T' are conjugate by the natural diffeomorphism of U and V. 

To complete this discussion describe the generating function of the billiard transformation 
T : V — > V. As before, use the position coordinates q in the linear space and the corresponding 
momentum coordinates p in the tangent space. The symplectic structure is dX, where A = p dq. Let 
(q,p) and (qi,Pi) be two points of W (so q, q\ 6 dM and \p\ = \pi\ = 1) such that cr(q,p) = (qi,Pi). 
Let H(q, q{) be the distance between q and q\. Then as in Section 1.2, ^ = —p, |^ = p\. Hence 

a*X — A = p\ dq\ — p dq = dH. 

fig. 8 

Thus H is the generating function of the billiard transformation, and in particular, we again 
see that this transformation is symplectic. The variational interpretation from Section 1.2 holds 
as well. 

1.6 Poincare's Recurrence Theorem 

We have seen that the billiard transformation is symplectic, and therefore, volume-preserving. 
A general fundamental property of volume-preserving transformation was discovered by H.Poincare. 

Theorem 1. Let T be a volume-preserving transformation of a manifold with a finite volume. 
Then for any neighbourhood U of any given point there exists a point x £ U which returns to this 
neighbourhood: T n x £ U for some positive n. The set of points in U that never return to U has 
zero volume. 

Proof. Consider the images of U : U, TU, T 2 U, ... They have equal positive volumes. Since the 
volume of the manifold is finite, some images intersect. Hence for k > I > one has: T k UnT l U ^ 0. 
Therefore T k ~ l U n U ^ 0. Let T k ~ l x = y for x, y € U. Then x is the desired point with n = k — I. 

Let V C U be the set of points that never return to U, that is V = U — U n >iT n U. Then V is 
measurable. For any n > 0,T n V C\V = % - otherwise a point of V will return to V, and therefore 
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to U. Hence the sets V,TV, T 2 V, ... do not intersect each other and, as before, one concludes that 
the volume of V equals zero. Q.E.D. 

One can strengthen the theorem by proving that almost all points of U return to it infinitely 
many times. One can also change the formulation of the theorem, replacing a manifold by a set with 
a finite measure, U by a measurable subset of nonzero measure and T - by a measure-preserving 
transformation. 

Poincare's theorem has numerous applications to classical mechanics and to billiards, in par- 
ticular. As an example, consider a point (x, v) in the phase space of a compact billiard M, that is, 
a position x of the billiard ball on the boundary dM and its inward unit velocity v. Then for any 
positive e there exists an e-close position y 6 dM and an e-close unit velocity u such that the ball 
(y, u) will eventually reflect in the boundary at a point e-close to x in the direction e-close to v. 

In the next section we will use the following corollary of the Poincare's theorem in its strength- 
ened form. 

Corollary 2. Let T be a measure-preserving transformation of a space M with a finite measure. 
Given a positive measurable function f on M, for almost all points x 6 M (in the sense of measure) 

oo 

J]/(T fc x)=oo. 

k=l 

Proof. Let M n be the set of points where f(x) > 1/n. Then M n is measurable and, according 
to the Poincare's theorem, almost all points of M n return to it infinitely many times. For such 
points the sum Ylh=i f(T h x) is clearly infinite. The result follows from the fact that M is the 
union of its subsets M n . Q.E.D. 

1.7 Billiard Transformation Revisited: Measure-Theoretic View-Point 

The aim of this section is to show that the billiard transformation is well defined off a set of zero 
measure in the phase space. Let M be a compact billiard table with a piecewise smooth boundary; 
let Ni,...,Nk be smooth components of the boundary, and assume that they have transversal 
pairwise intersections. Let V be the set of unit tangent vectors (x, v) with the footpoints x at the 
smooth part of the boundary and the inward directions v. The measure [i in V is the one induced 
by the symplectic structure, which is equivalent to the product of the Riemannian measure on the 
boundary and the Lebesgue measure on the unit sphere of vectors v in the tangent space T X M. 

The billiard transformation and the billiard flow for all times are defined at (x, v) if neither of 
the two "bad" things happen: the footpoint of T fc (x, v) belongs to one of the intersections Ni C\Nj, 
or the billiard ball (x, v) makes an infinite number of reflections in the boundary on a finite time 
interval. Since each intersection Ni n Nj has a positive codimension in the boundary dM, its 
measure equals zero. It follows that the set of points (x, v) for which the first possibility occurs is 
of zero measure. 

Consider the second possibility; let Q be the set of such points in V. 
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Lemma. /j,(Q) = 0. 

Proof. Define a positive function / on V. Given a point (x,v), let f(x,v) be the length of 
the geodesic segment through x in the direction of v until its first intersection with the boundary. 
Apply Corollary 1.6.2 to conclude that for almost all points (x,v) 

oo 

£/(T fc *) = oc. 

k=l 

If a point (x,v) belongs to Q then the sums 

J2T=i f{ Tkx ) sta y bounded as n — > oo. Hence 

fi(Q) = 0. Q.E.D. 

Thus the billiard flow is defined on a subset of full measure in the phase space. 

The second of the above mentioned possibilities, namely a trajectory that makes an infinite 
number of reflections on a finite time interval, can occur even for a plane strictly convex billiard 
whose boundary is three times diflerentiable (but this third derivative is unbounded). Such an 
example was constructed by B.Halpern. His construction involves rather meticulous estimates; its 
idea is, however, quite transparent. 

Consider a sequence of points p n of the unit circle that monotonically converges to a point of 
the circle (in the example the angular coordinate of p n is equal to n -1 / 2 ). The points p n are the 
consecutive points of reflection of a billiard trajectory in the billiard curve 7 to be constructed. 
Join the points p n to obtain a polygonal trajectory. The law of reflection determines the direction 
of 7 at the points p n . One constructs a small portion 7„ of 7 through each point p n and then 
connects j n in a smooth way to obtain 7. We omit the details; the interested reader is referred to 
[Ha]. It is worth mentioning that, in a sense, the Halpern's example is the best possible. Namely, 
Halpern proved the following theorem: if a billiard curve has a bounded third derivative and 
nowhere vanishing curvature, then the billiard flow is defined for all times. 

1.8 Complete Integrability and the Arnold— Liouville Theorem 

The billiard dynamics may vary from very regular to extremely chaotic. Here we discuss the 
most regular type of dynamical behavior: complete integrability, or integrability in the sense of 
Liouville. The considerations of this section are usually applied to continuous time dynamical 
systems; we will stick to a discrete time case. 

Definition. A symplectomorphism T of a symplectic manifold (M 2ti ,lo) is called completely 
integrable if there exist T-invariant smooth functions fi, f n (integrals) whose pair- wise Poisson 
brackets vanish and that are functionally independent almost everywhere on M (it means that 
their differentials are linearly independent in the complement of a set of zero measure) . 

Consider a nondegenerate level set P of the functions /1, /„. It is an n-dimensional manifold, 
whose tangent space is generated by the symplectic gradients of the integrals fi,...,f n . Since 
{fii fj} — ^(sgrad fi,sgrad fj) = it follows that P is a Lagrangian manifold. Such manifolds 
constitute a Lagrangian foliation (that is, a foliation whose leaves are Lagrangian manifolds) off 
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the set d fi A ... A d f n = 0. The symplectomorphism T preserves the foliation leaf- wise. Sometimes 
the existence of such a foliation is taken as the definition of complete integrability. 

Definition. An affine structure on a manifold is an atlas whose transition functions are affine 
transformations of the coordinate space (i.e., compositions of linear transformations and parallel 
translations). 

Lemma 1. A leaf of a Lagrangian foliation carries a canonical affine structure. 

Proof. Locally a foliation is a fibration; let tt : M — > Q be its local projection. Then 
smooth functions on the base Q lift to M and their symplectic gradients commute on M. If the 
differential of a function at a point x 6 Q vanishes then its symplectic gradient vanishes in the 
fiber 7r _1 (x) = P. Therefore a neighbourhood of each point of P is the local orbit of a locally free 
action of the additive group of the space T*Q. Whence the affine structure. Q.E.D. 

Said otherwise, the vector fields sgrad fi define a locally free action of R™ on P. 

The following result is a discrete version of the Arnold-Liouville theorem ( [Ve 1 , Ar 2] ) . 

Theorem 2. Let T be a completely integrable symplectomorphism. Then a compact connected 
component of a nondegenerate level set P of its integrals is diffeomorphic to an n-dimensional 
torus. In appropriate coordinates the restriction ofT to this torus is a translation: x — > x + a. 

Proof. Since P is connected the action of the group R n , defined in the previous proof, is 
transitive. Hence P is a quotient space of R n by a discrete cocompact subgroup, that is, a torus. 
Since T preserves the Lagrangian foliation leaf- wise, its restriction to a leaf preserves the affine 
structure therein. Since T commutes with the flows of the fields sgrad fi, it is a translation. 
Q.E.D. 

Thus the phase space of a completely integrable system is foliated by its invariant tori, and 
the transformation on each torus is simply a translation. 
The following corollary will be useful in the sequel. 

Corollary 3. If two completely integrable symplectomorphisms T and T' have the same foliation 
on invariant tori, then they commute: T T' = T' T. 

Proof. Restricted to an invariant torus both maps are translations in the same affine structure, 
whose definition depends on the Lagrangian foliation only. Translations commute; hence the result. 
Q.E.D. 

Let <j) = (01) ■•■) 0n) be coordinates on an invariant torus P in which T is a translation. A 
neighbourhood of P in M is foliated by invariant tori, parametrized by the values of the integrals 
/ = (/i,...,/ n ). Extend the coordinates <j> to these tori in a continuous way so that T is a 
translation in these coordinates on each torus. The coordinates (/, 4>) are not necessarily Darboux 
coordinates. It is possible, however, to choose a new set of variables I = I\, ...,/„, depending on 
/, so that u) = dl A d(f>. These coordinates are called the action-angle coordinates. 
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1.9 On KAM Theory 



Integrable systems are very exceptional: only a handful of examples is known. However, many 
important systems are small perturbations of integrable ones. A classical example is the solar 
system. The total mass of the planets is about 0.1% of the mass of the sun; therefore, in the first 
approximation, one can neglect the gravitational forces between the planets and consider only their 
attraction to the sun. The result is an integrable system, in which the orbit of each planet is an 
ellipse with a focus at the sun. 

The methods of classical perturbation theory, developed in celestial mechanics, are not quite 
satisfactory, because they lead to divergent series. The reason is that in computations one has to 
divide by " small denominators" - integer linear combinations of the frequencies of the unperturbed 
motion. If such a combination is small, the corresponding term in the perturbation series becomes 
too big. 

A breakthrough in this problem was made by A. Kolmogorov in 1954. He announced a 
theorem, that described the behaviour of Hamiltonian systems close to integrable, the theorem, 
later proved and generalized by V. Arnold and J. Moser. The whole theory is known as the 
Kolmogorov-Arnold-Moser (KAM) theory. We refer to [Ar 2, A-K-N, Mo 3,4, Mo-S, Pos, Bo] for 
a detailed discussion of the present state of the art in this rich area. 

We know from the previous section, that the motion in an integrable (discrete time) system 
reduces to a rotation on each its invariant torus. Let a{I) = (a±, a n ) be the translation vector 
as a function of the action variable /. Generically, the function a(I) is non degenerate, that is, 
det(^) ^ 0. In the nondegenerate case the frequencies ai,...,a n can be chosen as parameters, 
enumerating the invariant tori. These frequencies can be either linearly independent or linearly 
dependent over rational numbers. The former case is generic, in a sense: the corresponding tori 
constitute a set of full measure in the phase space. However, the latter case is unavoidable too: the 
corresponding tori are dense in the phase space. The situation here is analogous to the coexistence 
of rational and irrational numbers. 

For the sake of an illustration, consider a two-dimensional phase space, foliated by invariant 
circles. The transformation of each circle is a rotation. Consider a circle rotated through a tt- 
rational angle. A certain iteration of the transformation leaves each point of this circle fixed. As 
Poincare already noticed it is highly nontypical for an area-preserving map to have a whole circle of 
fixed points. Thus one expects such circles to be destroyed by a small perturbation of an integrable 
map. Likewise, the tori with linearly dependent frequencies most frequently vanish under a small 
perturbation of the transformation. 

However, if the frequencies are independent, and, moreover, their linear combinations are 
badly approximated by rational numbers in a certain precise sense, then the corresponding tori 
survive a small perturbation of a map in the class of symplectomorphisms (of course, the tori get 
perturbed slightly) . More precisely, consider an analytic symplectomorphism of an 2n-dimensional 
symplectic "annulus" T n x D n , exact with respect to the 1-form Idcj) and close to an integrable 
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one: 

(<f>, I) - (0 + a(I) + e 0(0, I, e), I + e 7 (</», /, e)); € T", / € D". 
Let the unperturbed map be nondegenerate: det(^) 7^ 0. 

Theorem. If the functions f3 and 7 are C r -smooth with r > 2n + 1, and e is sufficiently small, 
then there exist invariant tori in the "annulus", close to the unperturbed tori I = const. The 
measure of the complement of the union of these invariant tori goes to zero as e — ► 0. The motion 
on an invariant torus is conjugate to a translation. 

This result found in [Do 1,2] is an example of a theorem of KAM type. Many other versions 
can be found in the literature, in particular, concerning Hamiltonian vector fields and Hamiltonian 
systems in a neighbourhood of a fixed point. 

In the two-dimensional case invariant circles separate the phase space. Thus each orbit is 
bound to stay between two such circles and cannot escape to infinity. If the dimension is greater, 
then invariant tori do not separate the space anymore, and orbits can escape to infinity (a specific 
example of such a process has been found by V. Arnold; it is known now as Arnold's diffusion). 
This diffusion is very slow - its speed is of order exp(— l/e const ), where e is the parameter of 
the perturbation. This result is due to N. Nekhoroshev; it is one of the finest results of classical 
perturbation theory. 

1.10 Stochastic Properties of Dynamical Systems 

Completely integrable systems exhibit an extremely regular behavior. In this section we 
consider an opposite extreme: the maps that are, to a certain extent, chaotic. Our point of view is 
that of the ergodic theory, that studies maps of spaces with a measure. References are the books 
[C-F-S] and [Si 1]. 

Let M be a set with a cr-algebra of subsets and fi - a measure on this algebra. Assume 
that the measure is normed: n(M) = 1, and complete, that is, all subsets of measure zero sets 
are measurable. A dynamical system on M is a measure-preserving transformation T of M; this 
means that for any measurable set A one has: /jl(A) = n(T~ 1 A). An example to keep in mind is, 
naturally, the billiard transformation. 

The following definition is fundamental in ergodic theory. 

Definition. A dynamical system (M, T) is called ergodic if the measure of any T-invariant 
set equals either or 1. 

In an ergodic system any T-invariant measurable function / on M is a constant off a set of 
zero measure. Indeed, the set {x : f(x) < a} is measurable and T-invariant, hence its measure is 
equal to or 1. Mention that a completely integrable system is not ergodic: it has an abundance 
of invariant sets consisting of invariant tori. 

The following result is a consequence of the Birkhoff-Khinchin ergodic theorem for an ergodic 
dynamical system (M, T), that strengthens Corollary 1.6.2. 
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Theorem. For almost all points x 6 M the time average of a measurable function f equals its 
space average: 



n-1 

lim -Y / f(T k x)= / fdfi. 



k=0 

If / is the indicator of a measurable set (it equals 1 in the set and outside of it), then the 
left-hand side of the above formula is the average number of times the orbit of the point x visits 
this set. Thus, in an ergodic system, the trajectory of almost every point spends the time in a 
measurable set asymptotically proportional to its measure. Moreover, any two measurable sets are 
statistically independent in the sense that 



1 n— 1 

lim - V u.(T k A n B) = n.(A)u.(B). 



k=0 



Another stochastic property of dynamical systems, stronger than ergodicity, is mixing. 
Definition. A system (M, T) is called mixing if, for any two functions /, g € L 2 (M,fj,) the 
following equality holds: 



lim f f{T n x) g(x) d/z = / f(x) dp [ 
Jm Jm Jm 

Again if / and g are the indicators of two sets A and B then 

lim n{T~ n AnB) = fi(A) fi(B). 



g(x) do- 



lt means that the iterations of the map uniformly mix a measurable set in the space: after 
a number of iteration it becomes virtually impossible to distinguish the points that originally 
belonged to a given set. 

Assume that the set M is a compact topological space, the algebra of measurable sets consists 
of Borel sets (the minimal <r-algebra that contains open and closed sets) and the transformation T 
is a homeomorphism. 

Definitions. A homeomorphism T is called uniquely ergodic if there exists a unique normed 
Borel T-invariant measure on M. A homeomorphism T is called minimal if the trajectory 
{T n x, — oo < n < oo} of every point x is dense in M. A homeomorphism is called topologically 
transitive if there exists a point whose trajectory is dense. 

A uniquely ergodic homeomorphism is ergodic with respect to its unique invariant measure 
[i. Indeed if A is an invariant set with fi(A) ^ 0, 1 then one can define a new invariant normed 
measure: n'{B) = fj,(A n B) / n(A) . 

Clearly a minimal homeomorphism is topologically transitive. The properties of being minimal 
and uniquely ergodic are independent, although they often appear simultaneously in examples of 
dynamical systems. Minimal systems are, in a sense, similar to ergodic ones: they do not have 
nontrivial invariant closed subsets. If a dynamical system in a compact metric space is uniquely 
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ergodic, then the convergence of the time average to the space average in the ergodic theorem 
holds uniformly for any continuous function and for all (instead of only for almost all) points of 
the space. 

Finally we mention a statistical property stronger than mixing. 

Definition A measure preserving transformation T has K-property if for any measurable sets 
A ,Ai, ...,A k 

lim sup \n(A n B) - ji(.A Q ) ji{B)\ = 0, 

n— s-oo 

where supremum is taken over sets B from the a-algebra generated by the sets T^(Ai) with j > n 
and i = 1, k. 

Roughly speaking, this property means that the "present" is independent of the "past". "K" 
in "K-property" stands for Kolmogorov. 

1.11 Entropy 

The notion of entropy was introduced into physics by Clausius in the middle of the 19-th 
century, and about a century later it became a mathematical concept, first in information theory, 
due to Shannon, and then in ergodic theory, due to Kolmogorov. We discuss some very basic facts 
concerning entropy, referring to [Wa; Si 1,; Pet; D-G-S] for a detailed account. 

For a physicist the entropy of a system is a quantity proportional to the logarithm of the 
relative probability of its state. To motivate the definition, consider a simple model. Suppose a 
certain system consists of N identical particles, each of which can be at one of k different states. 
The number of ways to achieve the state in which exactly iV, particles are in the i-th state is equal to 
Nl , N ' Nk , ■ This is proportional to the probability of this state. Using Stirling's approximation In n\ ~ 
n In n — n this quantity approximately equals N In N — ^ TVj In TVj. Introducing the probability 
Pi = Ni/N of a particle to be in the i-th state, one gets for the probability per particle the 
following quantity: 

In N - ^pilnpi - InN^pi = - ^pilnpi. 

Given a set M with a probabilistic measure fi and its finite or countable partition on measurable 
sets £ = {Ai,A 2 , ...}, define the entropy of the partition 

#(6 = -E^) ln M^)- 

For a measure-preserving transformation T of M, define the partition T _n £ as {T~ n Ai, T~ n A 2 , ...}. 
Entropy of the transformation T with respect to the partition £ is 

h(£,T)= lim i J ff(£VT- 1 £V...VT-™ +1 0, 

n— s-oo 77, 

where £ Vr/ V ... denotes the partition into the pairwise intersections of the elements of the partitions 
£, r?, ... (this definition incorporates the lemma saying that the limit exists). The number /j(£, T) is 
a measure of the average uncertainty per unit time about which element of the partition £ the point 
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x will enter next, given its preceding history. A bad choice of £ might prove not very informative, 
though. One defines metric entropy of the dynamical system (M,T) as h(T) = sup h(£,T), taken 
over all finite or countable partitions £ with finite entropy. 

Metric entropy is an invariant of dynamical systems, that had made it possible to distinguish 
between the systems un distinguishable by the previously known invariants (such as the spectrum) . 
However the above definition can be rarely used for computations. The following theorem by A. 
Kolmogorov and Ya. Sinai makes the computations possible. A finite or countable partition £ is 
called a generator if VJ£,T 1 (£) generates the a-algebra of measurable sets. 

Theorem 1. If £ is a generator then h(T) = h(£, T). 

A useful corollary is that if there exists a one-sided generator £, i.e., V°^ T _l (£) generates 
the u-algebra of measurable sets, then the metric entropy of the transformation T vanishes. This 
agrees with the one's intuitution: the existence of such a partition means that the present and 
future of the system is completely determined by its past. We mention two properties of entropy. 
First, h(T n ) = \n\h(T) for an integer n; secondly, entropy of the product of two dynamical systems 
(Mi x M 2 ,T X x T 2 ) is equal to h(T{) + h(T 2 ). 

The next concept of the topological entropy of a transformation is independent of an invariant 
measure, unlike the metric entropy. Given a compact Hausdorff topological space M and its open 
cover U define N{U) to be the minimal cardinality of the subcovers of U. For two covers U 
and V let U V V be the cover, consisting of the pairwise intersections of their elements. Given a 
homeomorphism T of M define 

h top (U,T)= lim — In N(U V T~ X U V ... V T~ n+1 U), 

n— s-oo n 

and htop(T) = sup h top (U, T) over all open covers of M. 

The following result is known as the variational principle for topological entropy. 

Theorem 2. Let T he a homeomorphism of a compact metric space M . Then ht op {T) = sup h(T), 
where the supremum of metric entropy is taken over all T -invariant Borel probability measures on 
M. 

In particular, topological entropy is an upper bound for the metric one. 

For the later use we mention shift mappings. Consider a finite set {1, k} and let X be the 
set of sequences (xj), — oo < i < oo with Xi € {1, ...,k}. Put the discrete topology on the finite 
set and the product topology on X. Then X is a compact metrizable space. Let S be the shift 
transformation: S(x n ) = y n where y n = x n +i for all n. The shift is a homeomorphism of X whose 
topological entropy equals In A;. If Y C X is a closed shift invariant subspace then the dynamical 
system (Y, S) is called a subshift. Its topological entropy is equal to lim n ^ 00 ^ In N n , where N n is 
the number of distinct words of length n'mY. 

A Bernoulli shift is the shift of the space M of sequences {(x n )}, n 6 Z, x n € A where A 
is a probabilistic space and M has the product-measure. A Bernoulli shift is measure-preserving; 
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the above introtuced full shift corresponds to A = {1, k} and the uniform measure on A. 
Bernoulli shift is ergodic and mixing. 
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2. Convex Billiards 



The main object of study in this chapter are smooth convex billiards in the plane. We start 
with an elementary study of the billiard in ellipses; we use nothing but high-school mathematics 
here. These results are applied in the next section to some geometrical problems: "the most 
elementary theorem of Euclidean geometry" , which we prove in a non-elementary way, the problem 
of constructing a trap for a beam of light, and the illumination problem. Section 3 concerns the 
classical results by Jacobi on integrability of the geodesic flow in ellipsoids with applications to 
billiards in ellipsoids bounded by confocal quadrics. Section 4 deals with Birkhoff's conjecfture 
that the only integrable plane billiards are the ones in ellipses; "the mirror equation" appears 
there to play an important role in what follows. Section 5 gives a variational proof of the existence 
of Birkhoff's periodic trajectories of the billiard ball, the topic revisited in the next section from 
the point of view of symplectic geometry, and again in Section 10 in the framework of the theory 
of monotone twist maps. Section 7 concerns the relation between the set of length of periodic 
trajectories of the billiard ball and the spectrum of the Laplace operator in the billiard table 
with the Dirichlet boundary condition. Sections 8 and 9 discuss the existence and nonexistence of 
invariant circles of the billiard transformation: they exist if the billiard table is sufficiently smooth 
and strictly convex, and do not exist if it has a flat point. The methods used are those of the KAM 
theory and the theory of twist maps. The last section contains some isolated but nice results on 
smooth convex billiards not directly related to other topics of the chapter. 
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2.1 An Excursion to Elementary Geometry. Billiards in Conies 

The simplest billiard table is a circular one. There is not much to say about it: each trajectory 
makes a constant angle with the boundary and remains tangent to a concentric circle. The induced 
transformation on this tangent circle is a rotation through a fixed angle, that is, a translation. 
Translations of a circle deserve some attention; we delay their study until Section 3.1. 

fig. 9 

Before we proceed any further we introduce a new concept. 

Definition. A caustic of a plane billiard is a curve such that if a trajectory is tangent to it, 
then it again becomes tangent to it after every reflection. 

Thus the billiard in a circle has a family of caustics, consisting of concentric circles. 

The next case to consider is that of conies. Recall that an ellipse consists of points whose 
sum of distances to two given points is fixed; these two points are called the foci of an ellipse. An 
ellipse can be constructed using a string, whose ends are fixed at the foci - the method carpenters 
and gardeners actually use. A hyperbola is defined similarly with the sum of distances replaced by 
the absolute value of their difference; and a parabola is the set of points at equal distances from 
a given point (the focus) and a given line (the directrice). Ellipses, hyperbolas and parabolas all 
have second order equations in Cartesian coordinates. 

fig. 10 

The first result is the following optical property of ellipses. 

Lemma 1. A ray of light, emanating from one focus, comes to another focus after a reflection in 
the ellipse. Said otherwise, the segments, that join a point of an ellipse with its foci, make equal 
angles with the ellipse. 

Proof. Consider an extremal problem: given a line I and two points F 1 and F 2 on one side of 
it, find a point X on I such that the distance F\X + XF 2 is minimal. Solution: reflect Fi in the 
line and join with F 2 by a straight segment. The point of intersection with I is X. It follows that 
the angles made by F X X and F 2 X with I are equal. 

fig. 11 

On the other hand, X can be obtained as follows. Consider the family of ellipses with the 
fixed foci Fi and F 2 . Then X is the point where an ellipse from this family touches I for the first 
time. Hence X is the point of tangency of an ellipse with the foci Fi and F 2 and the line I. Q.E.D. 

To this we add that the billiard trajectory through the foci converges to the major axis of 
an ellipse. Indeed, consider consecutive segments of the trajectory. The point A 2 is closer to 
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the major axis than A 1 , etc. Hence there exists the limit of the sequence Ax,A 2 ,...; and, 
likewise, the limit of the sequence B 1 ,B 2 ,.... The segment A^B^ is itself a "to and fro" 
billiard trajectory: the billiard ball goes from A^ to B^ and back ad infinitum. Thus the segment 
AooBoo is perpendicular to the ellipse at both ends, and therefore is the diameter MN. 

fig. 12 

Likewise one proves the optical properties of a hyperbola and a parabola, shown in figure 10. 
These properties are extensively used in construction of various optical instruments. For example, 
if one puts a source of light in the focus of a parabolic mirror, then the reflected rays form a parallel 
beam - the property used in headlights' design. 

Ellipses and hyperbolas with the same foci are called confocal. In the appropriate Cartesian 
coordinates (x, y) they are given by the equation: 

x 2 v 2 

+ ,„ ; =1; < a < b. 



a? + \ b 2 + \ 

Here A is the variable parameter; for — b 2 < A < —a 2 the curve is a hyperbola, and for —a 2 < A 
it is an ellipse. 

The main result on elliptical billiards says that they are completely integrable. 

Theorem 2. An elliptic billiard table has a family of caustics, that consists of the confocal ellipses 
and hyperbolas. More precisely, if a segment of a billiard trajectory does not intersect the segment, 
joining foci Fx and F 2 , then all the segments of this trajectory do not intersect F 1 F 2 and are all 
tangent to the same ellipse with foci Fx and F 2 ; if a segment of a trajectory intersects F 1 F 2 , then 
all the segments of this trajectory intersect FiF 2 and are all tangent to the same hyperbola with 
foci Fx and F 2 . 

Proof. Let A Ax and AiA 2 be consecutive segments of a trajectory. Assume that A Ax does 
not intersect the segment FiF 2 (the other case is dealt with similarly). It follows from the optical 
property that the angles A AiFi and A 2 A\F 2 are equal. 

fig. 13 

Reflect F x in A A X to F[, and F 2 in A X A 2 to F 2 , and set: B = F{F 2 DA Ax, C = F/ ! F 1 nAxA 2 . 
Consider the ellipse with foci Fx and F 2 , that is tangent to ^o^i- Since the angles F 2 BAx and 
FxBAq are equal, this ellipse touches .Ao-Ai at the point B. Likewise an ellipse with foci Fx and F 2 
touches AxA 2 at the point C. One wants to show that these two ellipses coincide, or, equivalently, 
that F X B + BF 2 = F X C + CF 2 , which boils down to F[F 2 = FxF 2 . 

To this end one observes that the triangles F[AxF 2 and FxAxF^ are congruent: F[Ax = 
FxAx,F 2 Ax = F 2 Ax by symmetry, and the angles F[AxF 2 and FxAxF 2 are equal. Hence F[F 2 = 
FiFj, and the result follows. Q.E.D. 
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It might be instructive to compare this elementary proof with a more conceptual and general 
one we will give in the next section, concerning billiards in ellipsoids. 

As an addition to the theorem we remark that billiards bounded by different confocal conies 
are integrable as well, because their caustics are still confocal conies. 

fig. 14 

To finish this section, consider the problem of reconstructing a billiard table T from its convex 
caustic 7. The following "well known" string construction produces a one-parameter family of 
billiard curves (see [Por] and [Tu]). 

Lemma 3. Wrap a closed inelastic string around 7, pull it tight at a point and move the point 
around 7 to construct the billiard curve T. 

fig. 15 

Proof. Recall that the involute of a curve 7 is a curve, traced by the end of an inelastic 
(nonclosed) string, wrapped around 7. Fix a point of reference y on 7, and, given a point x outside 
of 7, draw the tangent segment xz to 7. Define a function f(x) as the length of the arc yz plus the 
length of the segment zx. Then the involute is a level curve of f(x). Since the string, that traces 
the involute, is inelastic, the tangential component of the velocity of the point x along zx vanishes. 
Hence the involute is perpendicular to the segment zx at each point x (see [B-G]). Therefore the 
gradient of f(x) is the unit vector in the direction of xz. 

Consider another involute, constructed by wrapping a string around 7 in the opposite direction, 
and let g{x) be the corresponding function in the exterior of 7. Then the curve V, described in 
the lemma, is a level curve of the function f(x) + g(x). Its gradient at x is the sum of two unit 
vectors in the directions of the tangent lines to 7. Hence the gradient makes equal angles with 
these tangent lines, and therefore, so does V. This is the desired billiard property. Q.E.D. 

In particular, wrapping a closed string around an ellipse produces a confocal ellipse - this is 
Graves theorem ([Be 1]). 

2.2 Three Geometrical Applications 

The first is "the most elementary theorem of Euclidean geometry" , formulated by M. Urquhart 
(see [El]): if AB + BF = AD + DF, then AC + CF = AE + EF. 

fig. 16 

This theorem has synthetic proofs (the author knows some); however, we will deduce it from 
integrability of elliptic billiards. The theorem states that if B and D belong to an ellipse with foci 



25 



A and F, then C and E belong to a confocal ellipse. 

Consider two confocal ellipses. The billiards therein have the same collection of caustics 
that consists of confocal ellipses. The family of rays, tangent to a caustic, is an invariant curve 
of the billiard transformation of the set of rays. Corollary 1.8.3 implies that the two billiard 
transformations commute. 

fig. 17 

Back to Urquhart's theorem. Construct the ellipses Y\ and T 2 with foci A and F, that contain 
the points B and C, respectively. The former ellipse contains D; one wants to show that the latter 
contains E. The billiards' reflection in T 2 , and then in Y\ sends the ray AB to DA. Hence the 
reflection, first in Ti, and then in Y 2 sends AB again to DA. This means that the points of 
intersection BF n Y 2 and AD n T 2 coincide. Hence E = BF n AD lies on F 2 . 

Urquhart's theorem has more or less straightforward generalizations (for instance, consider 
confocal hyperbolas instead of ellipses). We will not dwell on it. 

The next application is a construction of a trap for a beam of light, that is, a reflecting 
curve such that a collection of parallel rays of light, shone into it, gets permanently trapped. The 
question, asked in [Con], was answered by several authors (see [Guy] and [Fr]). We describe the 
trap, constructed by R. Peirone ([Pei]). 

fig. 18 

The curve 7 is a part of an ellipse with foci F\ and F 2 ; the curve Y is a parabola with focus F 2 . 
These curves are joined in a smooth way to produce a trap: it follows from the optical properties 
of conies, that a vertical ray, entering the curve through a "window", will tend to the major axis 
of the ellipse and therefore never escapes. 

It is tempting to change the problem slightly and to try to trap the set of rays sufficiently 
close to a given one in the space of rays (in other words, to allow the rays to make a small angle 
with a given one). However, in this case the trap does not exist, as readily follows from Poincare's 
recurrence theorem. Indeed, let U be the neighbourhood in the space of rays, one wants to trap. 
Close the window of the trap and consider the billiard inside this closed curve. There exists a ray 
in U, that returns back to U after a number of reflections. The only way to return to U is to get 
reflected in the part of the curve that constitutes the window. Hence this ray is not trapped in the 
first place. 

The third application concerns the illumination problem. Consider a room (a plane domain) 
with mirror walls; is it possible to illuminate it with a point source of light, that emits rays in all 
directions? 

fig. 19 
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An example of a room, that cannot be illuminated from any of its points, is shown in the figure 
(see [C-F-G]). The upper and the lower curves are half-ellipses with foci F 1 ,F 2 and Gi, G<z. Since a 
ray, passing between the foci, reflects back again between the foci, no ray can enter the shaded area 
from the one between the lines F\Fi and G1G2, and vice versa. A modification of this construction 
yields a region that, for any positive n, requires at least n sources for illumination. Likewise one 
constructs a bounded region, whose boundary is smooth at all but one point, that cannot be 
illuminated by any finite number of sources. However, for an everywhere smooth bounded region 
a finite number of sources will always do ( [Ra] ) . 

2.3 Geodesic Flow and Billiards in Ellipsoids 

The results of this section - integrability of the geodesic flow on an ellipsoid and of the billiard 
inside it - go back to the works of Chasles and Jacobi. In our exposition we follow [Mo 1,2, Ar 2]. 
For an approach via factorization of matrix polynomials see [Mo-V] . 

A quadric Q in a Euclidean space V with the scalar product ( , ) is determined by a self-adjoint 
operator A = A* : V — > V as follows: 

Q = {x€V: l -{Ax,x) = l). 

A quadric can be included in the following one-parameter families of quadrics. 
Definition. An Euclidean pencil of quadrics is a family of the form: 

{x £ V : ±((A-\E)x,x) = l}; 

a confocal family of quadrics is a family: 

Qx = {x€V: l -{ (A-\E)- 1 x,x) = l}, 

where A is a self-adjoint operator and E is the unit operator. 

The next figure shows an Euclidean pencil and a confocal family of conies. 

fig. 20 

Thus a confocal family consists of quadrics dual to the ones in an Euclidean pencil. One can 
easily verify that, in the plane, confocal quadrics have common foci. Unlike the plane case, in 
space there is no transparent geometrical description of confocal quadrics (however, see [H-CV] 
for a string construction due to O. Staude, the construction Hilbert considered one of the most 
remarkable results of 19-th century mathematics!). 

Given a quadric, one introduces coordinates in space. 

Definition. The elliptic coordinates of a point are the values of the parameter A, for which 
quadrics, confocal with the given one, pass through this point. 
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The geometry of confocal quadrics is described by the following two theorems, of which the 
first one justifies the term "elliptic coordinates". Fix an ellipsoid 

Q = {x: l -{A~ l x,x) = l} 

in an Euclidean n-dimensional space, whose axes have pairwise distinct lengths. 

Theorem 1 ( Jacobi) . A generic point of space is contained in exactly n quadrics, confocal with 
the given ellipsoid. Two confocal quadrics are perpendicular at their intersection points. 

Proof. A non-zero vector in space determines an affine hyperplane in the dual space that 
consists of the covectors whose values at the given vector equal 1. The statement of the theorem, 
formulated in terms of the dual space, reads: each affine hyperplane not through the origin is 
tangent to exactly n quadrics from a given Euclidean pencil; and the position vectors of the points 
of tangency are pairwise orthogonal. 

Let y be the original point in space; then the dual hyperplane is {x : (x,y) = 1}. Let the 
Euclidean pencil be ^( (A — XE) x,x) = 1. Introduce a new quadratic form B(x) = (Ax,x) — 
2(x,y) 2 , and consider a one-dimensional eigenspace L of this form with eigenvalue A. Then the 
quadratic form B—XE vanishes on L together with its differential. Consider the point of intersection 
z of L with the hyperplane (x,y) = 1. Since (B — XE) z = 0, the point z lies on the quadric 
|( (A — XE) x,x) = 1; and since the differential vanishes as well, this quadric is tangent to the 
hyperplane at z. 

Hence the points of tangency of the given hyperplane with quadrics of the Euclidean pencil 
\{ {A — XE) x,x) = 1 are eigenvectors of the quadratic form B(x). There are exactly n such 
vectors and they are pairwise orthogonal. Q.E.D. 

Theorem 2 (Chasles). A generic line in n-dimensional Euclidean space is tangent to (n — 1) 
distinct quadrics from a given confocal family. The tangent hyperplanes to these quadrics at the 
points of tangency with the line are pairwise orthogonal. 

fig. 21 

Proof. Project the space along the given line onto its (n — l)-dimensional orthogonal com- 
plement. A quadric determines a hypersurface in this (n — l)-space, the set of critical values of its 
projection (the apparent contour). If one knows that these hypersurfaces also constitute a confocal 
family of quadrics, the statement will follow from the previous theorem. 

To prove that it is indeed the case, one applies duality. Duality interchanges projections and 
sections. The apparent contour of a quadric is transformed by duality to the section of the dual 
quadric by the hyperplane through the origin, orthogonal to the direction of the projection. A 
confocal family is dual to a Euclidean pencil. The sections of quadrics from an Euclidean pencil 
by a hyperplane constitute a Euclidean pencil in it. Applying duality again, it follows that the 
apparent contours of quadrics from a confocal family constitute a confocal family. Q.E.D. 
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The next theorem says that the geodesic flow on an ellipsoid is completely integrable. 

Theorem 3 (Jacobi and Chasles). The tangent lines to a fixed geodesic on a quadric in Tri- 
dimensional Euclidean space are tangent to (n— 2) other fixed quadrics, confocal with the given one. 
The set of oriented lines, tangent to (n — 1) fixed confocal quadrics, is a Lagrangian submanifold 
in the space of rays in space. 

Proof. The proof consists of several steps. 

Step 1. Let I be an oriented line tangent to the given quadric Qo a t the point x. By Theorem 
2 it is tangent to (n — 2) confocal quadrics Q\, ...,Q n -2- Consider an infinitesimal rotation of the 
tangent line I along the geodesic on Q through x in the direction of /. Modulo infinitesimals of 
the second order, this line rotates in the 2-plane generated by I and the normal vector n to Q at 
x. By Theorem 2 the tangent hyperplane to the quadric Qi, i = 1, ...,n — 2, at the point of its 
tangency with I contains the vector re. Hence, modulo infinitesimals of the second order, the line I 
remains tangent to Qi, and thus remains tangent to each one of them. 

Step 2. This step consists of the following lemma. Let Q be a hypersurface in space. Consider 
the hypersurface of oriented lines, tangent to Q, in the symplectic manifold of oriented lines in 
space. Then the characteristic lines of this hypersurface consist of rays, tangent to a fixed geodesic 
on Q. 

To prove this statement we use the Melrose diagram from Section 1.5: 

X 

Y Z 
W 

U V 

s 

As before, X is the (co)tangent bundle of the space; Y consists of unit vectors and Z - of 
vectors with the footpoints on Q; U is the space of oriented lines and V is the (co)tangent bundle 
of Q; and £ consists of unit tangent vectors to Q. 

First, consider £ as a hypersurface in V, that consists of unit tangent vectors to Q. The 
characteristics of this hypersurface are identified with geodesies on Q. The same characteristic di- 
rections are obtained by restricting the symplectic structure of X to its codimension-3 submanifold 
S. And still the same characteristics £ has as a hypersurface in U. This hypersurface consists of 
oriented lines in space, tangent to Q. The statement follows. 

Step 3. Use the notation of Step 1. At a neighbourhood of the point of tangency of / with 
Qi fix a smooth function fa, whose level sets are the quadrics, confocal with Qi. Any line V close 
to / is tangent to a close confocal quadric Q\. Define a function on oriented lines, whose value 
at V is the value of fa on Q[. To show that a common level set of these functions is a Lagrangian 
manifold, one needs to show that the functions Fj pairwise Poisson commute. 

Fix a ray I, tangent to the quadrics Qi, and compute the derivative of F along the symplectic 
gradient of Fj at the point / of the space of rays. The field sgrad Fj is tangent to the characteristics 
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of the hypersurface Fj = const, that is, of the hypersurface, that consists of the lines, tangent to 
Qj. These characteristics are known to consist of the rays, tangent to a fixed geodesic line on Qj 
(Step 2). These rays are all tangent to Qi (Step 1); hence the function Fj does not change along 
the flow of sgrad Fj. Therefore {F h Fj} = 0. Q.E.D. 

Corollary 4. A trajectory of the billiard inside an ellipsoid in n-dimensional space is tangent to 
(n — 1) confocal quadrics. 

Proof. As explained in Section 1.1, the billiard flow inside an ellipsoid in n-dimensional space 
is the limit case of the geodesic flow on an ellipsoid in (re + l)-dimensional space, whose minor axis 
goes to zero. The result follows from the previous theorem. Q.E.D. 

Explicit formulas for the integrals of the billiard in an ellipsoid can be found in [Mo 2] . Let the 
ellipsoid be given by the equation (A -1 x,x) = 1, where A is a diagonal matrix with the diagonal 
elements Aj, and let (x,v) be a unit inward tangent vector, whose footpoint x lies on the ellipsoid. 
The following functions are invariant under the billiard transformation: 



These functions Poisson commute. 

Another corollary, one deduces from Theorem 3, concerns integrability of a billiard on a quadric 
in n-dimensional space, bounded by its intersections with confocal quadrics. 

Corollary 5. The tangent lines to a trajectory of such a billiard are all tangent to (n — 2) fixed 
confocal quadrics. 

Proof. Let 7 and 71 be two consecutive segments of a billiard trajectory on a quadric Q, and 
let Q' be the confocal quadric, whose intersection with Q is the boundary of the billiard. Denote 
by / and h the tangent lines to 7 and 71 at the reflection point. 



One knows that the tangent lines to the geodesic 7 are tangent to (n — 2) quadrics, confocal 
with Q. In particular, so is the line I. Consider the billiard inside Q' . Since Q' is orthogonal to Q 
this billiard transformation sends I to l\. By Corollary 4 the line l\ is tangent to the same (re — 2) 
quadrics. Hence the tangent lines to the geodesic 71 are tangent to the same quadrics as well. 



We refer to [Ve 2, D-G-K-R, C-S] for a discussion on billiards on a Euclidean sphere. The 
paper [Ve 2] by Veselov also concerns billiards in hyperbolic spaces. 

Let us mention that the lines of intersection of an ellipsoid in 3-space with confocal quadrics 
are its lines of curvature, i.e. the lines that have the eigendirections of the second quadratic form 
of the surface of the ellipsoid. Applying the string construction from Section 2.1, one concludes 
that these lines are analogous to confocal ellipses and hyperbolas in the the following sense: the 




fig. 22 



Q.E.D. 
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sum (or difference) of distances from all points of a line of curvature to two umbilic points of the 
ellipsoid is constant (umbilic points are the points where the principal curvatures are equal ; they 
are the singularities of the foliation by the lines of curvature) - see [H-CV] . 

fig. 23 

In conclusion we remark that the geodesic flow on an ellipsoid was first integrated by Jacobi 
using elliptic coordinates to separate the variables in the equation of motion, which is now called 
the Hamilton-Jacobi equation. 

2.4 Birkhoff's Conjecture 

We know that the billiard inside an ellipse is integrable: its caustics are confocal ellipses and 
hyperbolas. The partition of the phase cylinder into invariant curves of the billiard transformation 
is shown in the figure (the oo -shaped curve corresponds to the rays through the foci). Notice the 
topological difference between the cases of an ellipse and a circle. 

fig. 24 

G. Birkhoff conjectured that ellipses are characterized by integrability of the billiard transfor- 
mation. 

Conjecture. If a neighbourhood of a strictly convex smooth billiard curve is foliated by caustics, 
then the curve is an ellipse. 

Several attempts had been made to prove this conjecture, but, so far, it remains open. A 
partial result was obtained recently by M. Bialy [Bia]. In our exposition we combine the ideas of 
Bialy and M. Wojtkowski [Wo 1]. 

We start with two observations of independent interest. Let H be the function on the phase 
cylinder V, whose value at the point (x,v) is the length of the trajectory of the "ball" (x,v) until 
it hits the billiard curve. Recall that V has an invariant measure fi, introduced in Section 1.2. 

fig. 25 

The following result is well known in integral geometry. 
Lemma 1. Let A be the area of a convex billiard table. Then 

H dfi = 2vrA 

Jv 

Proof. Recall (Section 1.3) that the set of oriented lines in the plane has a natural measure, 
associated with the form d p A d <j>, where p is (±) the distance from the origin to the line and 4> is 
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its angle with a fixed direction. This measure is equal to \x under the identification of the cylinder 
V with the set of lines U, that intersect the billiard curve. So 

P P P '2 71 

H dp = H dp dcf) = A d(f>= 2ttA; 
Jv Ju Jo 

the second equality is due to the fact, that for a fixed direction (j> the integral J H dp is the area. 
Q.E.D. 

The next result is the classical "mirror equation" of geometric optics (see, e.g., [Por]). Consider 
a caustic of a billiard and introduce the variables a,b,9, as shown in the figure. Let K be the 
curvature of the billiard curve at the point x. 

fig. 26 

Lemma 2. 

1 1 _ 2K 
a b sin 6 



Proof. Replace the billiard curve by its circle of curvature at the point x, which has the 
second order tangency with the curve. Consider a nearby point x', seen from the center of the 
circle at the angle of e. Draw the tangent segments to the caustic from x', and denote the angles 
xyx' and xzx' by a and f3. 

fig. 27 

It follows from figure 27 that the beam of light, emanating from the point y, focuses at the 
point z after reflection in the curve. 

The Sin theorem for the infinitesimal triangles xyx' and xzx' yields: 

sin# aK sin# (3K 
a e ' b e 

Hence 

1 1 _ (a + 0) K 
a b e sin 6* 

It remains to show that a + (3 = 2e. Consider the closed path yxx'zxx'y; being homotopic to a 
twice traversed circle, its total rotation equals 4tt. This rotation consists of the exterior angles at 
the vertices and twice the turn of the arc xz', that is, 2e. Thus 

9 + ip + (vr - (3) + (vr - 9) + (vr - tp) + (vr - a) + 2e = 4vr, 

or: a + (3 = 2e. Q.E.D. 

Now we are in a position to prove a particular case of Birkhoff 's conjecture. 
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Theorem 3. If a billiard table is foliated by smooth closed convex caustics so that almost every 
trajectory is tangent to a caustic, then the table is a disc. 

Proof. Fix a caustic. Given a point x on the billiard curve, let v be the unit tangent vector 
in the direction of xz. Denote the billiard transformation by T. Lemma 2 reads: 

2 sing _ 4b(x,v) (H(x,v) - b(T(x,v)) 
K{x) ~ b{x,v) + {H{x,v) -b{T(x,v))' 

In view of the inequality between the harmonic and the arithmetic mean the right-hand side is not 
greater than H(x,v) + b(x,v) — b(T(x,v)). Integrate both sides over the phase space. Since the 
measure is T-invariant, 

/ (H(x,v) + b(x,v) -b(T(x,v)) dfj,= / H(x,v) du. = 2ir Area. 
Jv Jv 

Since fj, = sin 6 d9 dl, where / is the length parameter (Section 1.2), the integral of the other side 
equals 

/ / 2 -WV dldf, = 'l W) il ' 

where L is the length of the billiard curve. By the Cauchy-Schwartz inequality 




Since the second of these integrals equals 2ir, one concludes: 

I? 

— < 2ir Area. 
2 ~ 

This inequality is opposite to the isoperimetric one, hence it is actually an equality, and the curve 
is a circle. Q.E.D. 

Theorem 3 is slightly weaker than Bialy's result. His theorem assumes that the phase cylinder 
is foliated by continuous closed noncontractible invariant curves, with the same conclusion that 
the billiard table is a disc. Some additional work is needed to handle this continuous case; we will 
not dwell on it. 

A result somewhat related to Birkhoff's conjecture was obtained by E. Amiran ([Am]). Con- 
sider strictly convex smooth integrable billiards with the following evolution property: if any caustic 
is taken as a new billiard curve, then this new billiard is also integrable and shares its caustics 
with the original one. This evolution property holds for elliptic billiards; it follows from Amiran's 
work that the converse is also true. 

A different version of integrability is the analytic one: a piecewise smooth billiard is analytically 
integrable if there exists an invariant function, continuous in the position variable and analytic in 
the velocity variable. S. Bolotin [Bol] proved that such a billiard curve consists of pieces of straight 
lines and conies. The methods of the proof are those of algebraic geometry. 
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Notice that, although Birkhoff's conjecture is not proved yet, integrable billiards certainly 
constitute a "small" subset in the space of convex billiards. More precisely, it is a set of the first 
Baire category (a countable union of nowhere dense sets) in an appropriate topology in the set of 
analytic - see [K-T] , and smooth convex curves - see [La 3] . 

2.5 Periodic Trajectories 

Given a C 1 smooth strictly convex billiard curve, does the billiard map have periodic trajec- 
tories, that is, points in the phase space that return back after a number of iterations? These 
periodic trajectories are inscribed polygons, whose consecutive sides make equal angles with the 
billiard curve. We distinguish star-shaped n-gons with rotation r (i.e., the sum of the exterior 
angles at the vertices, divided by 2ir). These polygons are obtained by choosing n points x\, x n 
on the curve in the clockwise order and connecting x\ to x r+ \ to X2 r +i-, etc. 

fig. 28 

The answer to the above question is given by the following theorem by G. Birkhoff ([Bi 1]). 

Theorem 1. For any n > 2 and r < n/2, coprime to n, there exist two geometrically distinct 
n-periodic trajectories with the rotation r. 

Proof. Consider the set M of inscribed n-gons xi, ...,x n , where Xi is a point on the curve. 
M is an n-dimensional torus. Let H be the perimeter length function |xix r _|_i| + \x r+ iX2 r +i | + .... 
This function is smooth off the singular set Mq = L){xi = Xi+i} (we adopt the convention that 
n + 1 = 1), and at a generic point of M the function has the absolute value type singularity. 
We learned in Section 1.2 that critical points of the function H are billiard trajectories (this is 
Maupertuis' Principle from classical mechanics -see [Ar 2] - as applied to billiards). 

First we prove the easy part: for any n there exists an n-periodic trajectory. M being compact, 
H attains its maximum on it. This maximum is not attained on M : the perimeter of a /c-gon 
with k < n can be increased by introducing a new vertex. Thus this maximum corresponds to a 
faithful n-gonal trajectory. 

fig. 29 

To prove the result in full, consider the set of inscribed star-shaped n-gons with rotation r. 
Abusing the notation, call it M. This set is the product of a circle and an (n — l)-dimensional 
disc, and its boundary consists of degenerate polygons from M . The function H has at least n 
maxima in M, obtained by a cyclic permutation of the vertices of geometrically the same polygon 
of the maximal perimeter in M. 

Use the min-max (or the Buridan's Ass) argument to show, that there are other critical points 
of H inside M. Connect two maxima by a curve inside M and consider the minimum of H on it. 
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Take the maximum of these minima over all such curves. This is also a critical point of H, other 
than the maxima. A subtle point is to establish its existence. It follows from the fact, that one 
does not need to come close to the boundary M , since the function H increases as one moves from 
the boundary - see somewhat technical details in [C-S] or [K-T]. 

fig. 30 

The reason to consider r coprime to n, is that, otherwise, the above argument might yield 
a trajectory which is a polygon with fewer vertices, traversed by the billiard ball several times. 
Q.E.D. 

It follows, that there are at least 4>(n) distinct n-periodic orbits, where <j){n) is the number of 
integers, less than n and coprime to n. 

The following particular case may shed some light on the proof. Consider 2-periodic "to 
and fro" trajectories, that is, the chords of the billiard curve, perpendicular to it at both ends. 
One such curve is easily found: it is the diameter, i.e., the greatest chord of the curve. Since 
the billiard curve is strictly convex, there is a unique chord of maximal length in each family of 
parallel chords. Thus one has a one-parameter family of these longest chords, parametrized by 
their directions, whose initial element is the diameter, and whose terminal element is the same 
diameter with the end-points interchanged. In this family there exists a shortest chord, and this 
chord is perpendicular to the curve at both ends. This is the desired second 2-periodic billiard 
trajectory. 

fig. 31 

One naturally wonders whether there are any higher-dimensional generalizations. The situa- 
tion with 2-periodic orbits is particularly clear: a convex smooth body in n-dimensional space has 
at least n chords, perpendicular to the boundary at both end-points (due to N. Kuiper. Proof: 
assign to a line in space the length of the orthogonal projection of the body on this line. This is 
a function on projective space whose critical points correspond to the desired chords). The case 
of trajectories with a greater number of links is much harder, and not much is known about it. 
Recently I. Babenko proved that a three-dimensional smooth strictly convex billiard has at least 
n distinct periodic trajectories for any prime number n ([Bab 1]). His approach is similar to the 
one in dimension two, and it involves the Morse theory of the length functional on the space of 
inscribed polygons. An obvious difficulty is the lack of a rotation number in this setting. 

After the existence of periodic trajectories is established, one would like to learn about their 
stability. The billiard transformation T is a mapping of the phase annulus; its n-periodic orbits 
are fixed points of the map T n . 

Definition. A fixed point x of a map F is stable (in the sense of Lyapunov), if for any 
neighbourhood U of x there exists a neighbourhood V such that the orbit L)F l (V), — oo < i < oo, 
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is contained in U. 

Given an n-periodic trajectory of the billiard ball, consider the corresponding fixed point of 
the mapping T n . The derivative of T" at a fixed point is an area-preserving linear transformation 
of the plane; hence its determinant equals 1. Generically two cases are possible: the eigenvalues are 
reciprocal distinct reals, or they are distinct conjugate complex numbers with the absolute values 
of 1. In the former case (called hyperbolic) the linear map is a hyperbolic rotation; the fixed point 
is not stable (even in the linear approximation). In the latter case (called elliptic) the linear map is 
a rotation, and the fixed point is stable in the linear approximation. It follows from KAM theory, 
that in the general position, there exist invariant curves of the map T n near the fixed point. Thus 
an elliptic periodic trajectory is generically stable. 

fig. 32 

Without going into details we mention, that the maximal length Birkhoff 's periodic trajectory 
is always unstable. The stability condition for a 2-periodic orbit reads as follows. Let I be the 
length of the segment of such a trajectory, and ri,r 2 be the radii of curvature of the billiard curve 
at its end-points, which we allow to be negative as well. 

Lemma 2. A 2-periodic orbit is elliptic if and only if 

^±^>0 and {l ~ ri){l ~ r2) > 0. 
ri r 2 ri r 2 

Accordingly, the major axis of an ellipse is an unstable, and the minor one is a stable 2-periodic 
trajectory of the billiard ball. We refer to [Wo 2] for the proof of the lemma. 

Since a periodic trajectory is a critical point of the length functional, another approach to the 
stability problem is via the study of the Hessian matrix at this critical point. The eigenvalues of 
the linearization of the appropriate iteration of the billiard transfromation at its fixed point can 
be expressed in terms of the Hessian - see [Ma-M] and [K-T] (and [Kl] for a similar question in 
the study of closed geodesies in Riemannian manifolds). An application to 2-periodic trajectories 
is given in [Bar]. 

In conclusion of this section we mention the works by Ph. Levallois and M. Tabanov ([L-T, 
Lev]) and V. Donnay ([Don 1]) on non-integrability of billiards obtained by small perturbations of 
an ellipse. The general mechanism responsible for this non-integrability had been discovered by H. 
Poincare in his study of the three body problem; it is called the splitting of separatrices (see, e.g., 
[Ar 2]). 

In an elliptic billiard the major axis is a hyperbolic 2-periodic trajectory. The set of rays 
through one or another focus constitute a curve, invariant under the billiard transformation, that 
connects the two points of this 2-periodic trajectory (this curve is called a separatrix) - see figure 
24. Any point of the separatrix goes to one of the two points (B in figure 33) under positive 
iterations of the billiard map, and to another one (point A) under negative ones. 



36 



fig. 33 



Make a small perturbation of the billiard curve. The 2-periodic hyperbolic trajectory will 
persist (it will still be the diameter of the curve), however the above constructed curves for the 
new points A and B will not necessarily coinside. More precisely, let Sa be the curve consisting 
of points x with T~ n x — > A as n — > oo, and Sb = {x \ T n x — > B, n — > oo}. If the curves Sa and 
S's intersect once, they will intersect infinitely many times because both are invariant under the 
billiard map. The two curves constitute a complicated web, whose existence does not agree with 
integrability. 

Levallois and Tabanov analyzed the splitting of separatrices in the case when the perturbed 
billiard table is the following curve of degree four: 

x = acosa, y = 6sina(l + ecos 2 a), 

e being a small parameter. They proved that for all sufficiently small e the separatrices intersect 
transversally, and estimated the angles made by them at intersection points. V. Donnay obtained 
a similar result for a small perturbation of an ellipse in a neighbourhood of one point; this pertur- 
bation changes the curvature, but preserves the point and the tangent direction at it. 

2.6 Periodic Trajectories, Poincare's Last Geometric Theorem and Sym- 
plectic Topology 

Another approach to the existence of periodic trajectories consists in using a theorem on fixed 
points of area-preserving maps of an annulus. In this section we describe this approach and put it 
into the broader perspective of symplectic topology. 

Recall that the phase space of the billiard transformation T inside a smooth strictly convex 
region is an annulus with the coordinates (t, a), where t is the length parameter along the billiard 
curve and a is the angle made by the velocity vector with the boundary. Assume for convenience 
that the length of the billiard curve is 2ir. The two boundary circles of the phase annulus are a = 
and a = 7r; these curves are pointwise preserved by the transformation T. 

Fix a point x on the billiard curve and consider the unit tangent vectors, whose footpoint is x. 
This set is the vertical segment, shown in figure 34. The image of this segment under T is a curve, 
that makes one complete turn around the annulus. One can lift T to a map T of the universal 
covering of the annulus, which is an infinite strip. The choice of T is made by requesting that it 
pointwise fixes the lower boundary a = 0. Then the upper boundary is translated by T through 
2ir. In down-to-earth terms, one says that T fixes one boundary and rotates another one in the 
positive direction through 2ir. 

fig. 34 

Let R be the rotation of the annulus in the negative direction through 2tt (that is, a negative 
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translation of the covering). Then n-periodic orbits of T with rotation number r are fixed points of 
the map T n R~ r . Notice that this map translates the boundary circles in the opposite directions. 
The existence of two fixed points follows from Poincare's Last Geometrical Theorem (H. Poincare 
announced it shortly before death; the proof was given by G. Birkhoff in 1917). 

Theorem. An area-preserving transformation of an annulus, that moves the boundary circles in 
the opposite directions, has at least two distinct fixed points. 

Modern proofs of this theorem use the methods of symplectic topology we will discuss later in 
this section. We chose to give here the elegant topological proof by Birkhoff ([Bi 1]), which, from 
the modern point of view, might seem somewhat of an anachronism. More precisely, we show that 
the transformation in question has fixed points (this is the most nontrivial part of the argument). 

Proof. To fix ideas, let the universal covering of the annulus be the strip < y < 1, and let 
T move the lower boundary to the right and the upper one - to the left. Assume that T has no 
fixed points, and let e be smaller than the distance between any point and its image under T (it 
exists due to compactness of the annulus). Denote by S e the vertical translation of the plane, in 
which the strip lies, through e, and set: T S e = T t . 

Consider the strip < y < e. Its images have equal areas, and their interiors are disjoint. 
Hence there is an image of this narrow strip that intersects the upper boundary y = 1 ; let k be the 
least number of iterations needed. Let Pk be the point of the upper boundary of the fc-th image 
of the e-strip with the greatest y-coordinate, and let Pq = T~ k (Pk) be the corresponding point of 
the line y = 0. Set T*(P ) = Pi- Join Po an d Pi by a segment and consider its consecutive images 
under T e . They constitute a simple arc from P to 

fig. 35 

For a point Q of the strip consider the vector v(Q) = (Q, T e Q). HQ is on the lower boundary, 
its direction is close to that of the positive x-ray, and for Q on the upper one its direction is almost 
opposite. Let Q move along the above constructed arc from Pq to Pk-i- Then, for a sufficiently 
small e, the rotation of v(Q) is the same as that of the tangent vector to the arc (one adds the 
external angles at the corners), that is, almost equals tt. Therefore the rotation of v(Q) almost 
equals tt for any continuous motion of Q from the lower to the upper boundary. This conclusion 
"survives" the limit e — ► 0, in which case one gets rid of the word "almost". 

Consider now the inverse map T -1 , that moves the lower boundary to the left and the upper 
- to the right. By entirely analogous argument, the rotation of the vector u(Q) = (Q, T e _1 Q) 
equals (—tt) as Q moves from the line y = to y = 1. But u(Q) = —v(Q), so the rotations of these 
vectors are equal. The contradiction means that T has a fixed point. Q.E.D. 

The theorem of symplectic topology, that generalizes (and implies) Poincare's theorem, reads 
as follows. Consider a transformation T of a two-dimensional torus: T(x,y) = (x + f(x,y),y + 
g(x, y)), that preserves the area form dxAdy, and assume that it also preserves the center of gravity: 
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the average values of / and g are equal to zero. Then T has at least 4 fixed points, counted with the 
multiplicities, and at least 3 of them are geometrically distinct. Given an annulus transformation, 
as in Poincare's theorem, one constructs a torus transformation, pasting together two copies of the 
annulus along two narrow strips (see [Ar 3]). 

In the case of an 2re-dimensional standard symplectic torus the numbers of fixed points of 
a symplectomorphism, that preserves the center of gravity, are equal to 4 n (counted with the 
multiplicities) and 2re + 1 (geometrically distinct). This theorem by Conley and Zehnder ([C-Z]), 
published in 1983 and conjectured, among other things, by V. Arnold in the 60-s, was a break- 
through result in symplectic topology. A more general conjecture states that a symplectomorphism 
of a compact symplectic manifold, symplectically homological to the identity (that is, the time-1 
transformation of a Hamiltonian vector field with a time-dependent Hamiltonian function), has 
at least as many fixed points as the least number of critical points of a smooth function on this 
manifold. This conjecture is proved in many cases, in particular, for surfaces and toric varieties. 

Symplectic topology is a very exciting subject, but a more detailed discussion would lead one 
too far away from billiards. The interested reader may consult the surveys [Ar 1,4; Gro; Sik; Ben]. 

2.7 Length Spectrum and Laplace Operator 

The existence of periodic trajectories being established, what can be said about their lengths? 
This is a counterpart to the question one asks in differential geometry concerning the lengths of 
closed geodesic lines in a Riemannian manifold; and indeed, there is a strong similarity between the 
results in both areas. We will outline some results on the set of lengths of the closed trajectories 
in a convex billiard, the set that is called the length spectrum of a billiard. 

To start with, let us explain the symplectic meaning of the length spectrum, following V. 
Guillemin and R. Melrose ([G-M]). Let T be an exact symplectomorphism of an exact symplectic 
manifold (M, uS). This means that go = da for a 1-form a, and that the closed 1-form T*a — a 
is exact, i.e., equal to dF for a generating function F. Let x be an re-periodic point: T n x = x. 
Assign to this point the number 

n 

L{x) = Y J F{Tx). 

i=i 

This number depends on the choice of the generating function, determined up to an additive 
constant. One normalizes F in a natural way: say, by requiring that its average vanish, or that F 
vanish on a boundary component (in the case of billiards) . Notice that L{x) depends on the periodic 
orbit, but not on a particular choice of the point: L{x) = L{Tx). Finally L{x) is independent of a 
change of a by a coboundary: if a' = a + d(j), then F' = F + (poT — <j) and 

L'(x) = F{T l x) + (f>(T i+1 x) - <j)(T l x) = ^ F(T ! x) = L(x). 

i 

In particular, L(x) is a symplectic invariant of a periodic orbit for a simply connected symplectic 
manifold. 
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In the billiard setting the symplectic phase manifold is the unit disc subbundle of the 
(co)tangent bundle of the boundary of the billiard table, and the generating function is (neg- 
ative) the length of the trajectory segment between two consecutive reflections. This function 
vanishes on the boundary of the phase space, that consists of tangent vectors to the billiard hy- 
persurface. Thus the symplectic invariant of a periodic orbit specializes to the length of a closed 
billiard trajectory. 

S. Marvizi and R. Melrose ([M-Me]) studied the asymptotics of the lengths of n-periodic 
billiard trajectories in a smooth strictly convex plane domain as n — > oo. Let L n be the supremum 
and l n - the infimum of the perimeters of simple billiard ra-gons. 

Theorem 1. 

lim n k (L n - l n ) = 

n— >oo 

for any positive k. Moreover, L n has an asymptotic expansion as n — > oo: 

CO 

where I is the length of the billiard table and Ci are constants, depending on the curvature of the 
table. 

Y. Collin de Verdiere ([Col]) similarly studied periodic trajectories near a stable 2-periodic 
billiard orbit. 

The idea of the proof is to deal with the billiard transformation T as if it were integrable. More 
specifically, there exists a smooth function / in a vicinity of the boundary of the phase annulus, 
such that T _1 o exp sgrad (f 3 ^ 2 ) is a smooth map, that fixes this boundary to infinite order. In 
particular, the Taylor expansion of / is uniquely determined along the boundary. The appearance 
of the exponent 3/2 is due to the fact, that T has a square root type singularity at the boundary. 
For the integrable map exp sgrad (f 3 ^ 2 ) one expresses the length spectrum in terms of the Taylor 
coefficients of the function /, which depends on the geometry of the billiard curve, and the result 
follows. 

In the case, considered by Collin de Verdiere, one deals with a vicinity of an elliptic fixed 
point of an area-preserving map. If the derivative of the map is a rotation through a 7r-irrational 
angle, then the map is approximated to infinite order by an integrable map: a rotation through a 
variable angle, depending on the radius (Birkhoff's normal form - see [Ar 2, Mo-S]). After this the 
proof proceeds along similar lines. 

A remarkable relation exists between the length spectrum of a convex billiard and the spectrum 
of the Laplace operator in it with Dirichlet boundary condition: 

Af = Xf in M, f\ dM = 0. 

From the physical point of view, the eigenvalues A are the eigenfrequencies of the membrane M 
with a fixed boundary. 

Roughly speaking, one can recover the length spectrum from that of the Laplacian. More 
precisely, the following theorem of K. Anderson and R. Melrose ([A-M]) holds. 
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Theorem 2. The sum 

cos (ty/^\ ) 

Xidispec A 

is a well-defined generalized function (distribution) of t, smooth away from the length spectrum. 
That is, if I > belongs to the singular support of this distribution, then there exists either a 
closed billiard trajectory of length I, or a closed geodesic of length I in the boundary of the billiard 
table. 

A relation between the Laplacian and the length spectrum is, of course, not a coincidence. 
Geometric optics is not a very accurate description of light. In wave optics light is considered as 
electromagnetic waves, and geometric optics gives a realistic approximation only when the wave 
length is small. This small-wave appproximation is based on the assumption that the waves are 
locally almost harmonic, while their amplitudes change slowly from point to point. The substitution 
of such a function into the corresponding PDE's gives, in the first approximation, the equations 
of wave fronts, that is, of geometric optics (see [G-S 1,2]). Precise results in this direction are 
not easy to prove, and even the formulations are rather technical. See, for instance, the works of 
Lazutkin ([La 2,3]), in which approximations of the Laplacian's eigenvalues are constructed from 
caustics of the corresponding billiard. 

2.8 Existence of Caustics 

An important property, that makes it possible to apply the KAM theory to the billiard trans- 
formation, is that it is close to an integrable map near the boundary of the phase cylinder. Let 
T(t,a) = (t 1 ,a\). One can express the values of t\ and ct\ in terms of t, a, and the curvature 
of the billiard curve. We omit these straightforward, but rather cumbersome, computations, and 
state the answer ([La 1]): 

h=t + 2p(t)a + ^p(t)p'(t)a 2 + f(a, t)a 3 , 
2 

oii = a — -p'(t)a 2 + g(a,t)a 3 . 

Here p(t) is the radius of curvature of the strictly convex billiard curve, / and g are some smooth 
functions. Using ideas from the geometrical optics Lazutkin found a smooth change of variables 
x(t, a),y(t, a) with y(t, 0) = that reduces the billiard map to a simple form: 

xi =x + y + f(x,y)y 3 , y x = y + g(x,y)y A . 

It follows that near the boundary curve y = the map is a small perturbation of the integrable 
shear map (x,y) — > (x + y,y). It is also area-preserving, hence the KAM theory is applicable. One 
arrives to the following result by Lazutkin ([La 1]). 
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Theorem 1. If the billiard curve is sufficiently smooth and its curvature never vanishes, then 
there exists a collection of smooth caustics in a vicinity of the billiard curve, whose union has a 
positive area. 

Originally this theorem asked for 553 continuous derivatives of the billiard curve; later R. 
Douady using results of Russman and Herman reduced the number to 6 ([Do 1,2]). 

Notice that a smooth closed caustic in a vicinity of the billiard curve corresponds to an 
invariant circle of the billiard transformation in the phase cylinder, homotopic to its boundary 
cirle. This invariant circle separates the cylinder into two invariant subsets of positive measure. 
Therefore strictly convex smooth billiards are not ergodic. 

Let us mention that caustics do not have to be convex curves. For example, consider a curve 
of constant width. It has a chord in any direction, perpendicular to the curve at both ends. These 
chords are 2-periodic billiard trajectories, and their envelope is a caustic. This caustic is the 
involute of the billiard curve (see [B-G]); it necessarily has cusps, that correspond to the critical 
points of the curvature of the billiard curve. This example is considered in [Gu-K 1]. Another 
example from the same paper is the caustic, given by the equation x 2 / 3 + y 2 / 3 = 1, called the 
astroid. Using a modification of the string construction, one recovers a one-parameter family of 
convex billiard tables from this caustic. 

fig. 36 

The situation with caustics of higher-dimensional billiards is very different. Given a smooth 
strictly convex billiard table in space, its caustic is a hypersurface that enjoys the same property: 
a ray tangent to it, remains tangent after the reflection in the billiard hypersurface. An ellipsoid 
has a collection of caustics, the confocal ellipsoids. M. Berger proved the converse. Suppose that a 
billiard hypersurface has a caustic. Then the collection of rays through a point of the hypersurface, 
tangent to the caustic, is a symmetric cone, whose axis is perpendicular to the billiard hypersurface. 
M. Berger's result reads (see [Be 2,3]): 

Theorem 2. If M is an open C 2 - smooth hypersurface and N is an open hypesurface such that 
for each point x 6 M there exists a hypercone with the vertex at x, tangent to N, symmetric with 
respect to its axis, which is perpendicular to M, then M is a part of a hyperquadric and N is a 
part of a confocal quadric. 

Notice that this result is local: the existence of a piece of a caustic already proves to be a very 
rigid condition, in sharp contrast with the plane case. Also notice that this theorem does not imply 
the higher-dimensional version of Birkhoff 's conjecture: only billiards in ellipsoids are integrable. 
Indeed, if the billiard transformation of the space of rays has an invariant hypersurface, this 
hypersurface does not necessarily consist of rays tangent to some hypersurface in the configuration 
space. The proof of Berger's theorem is computational and makes use of the classical differential 
geometry. Its key ingredient is the mirror equation from Lemma 2.4.2. 
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To this we add that, according to [Gru], most convex billiards (in the sense of Baire's category; 
the topology in the set of convex sets being determined by the Hausdorff metric) have no convex 
caustics. 

2.9 Twist Maps, Birkhoff 's Theorem and Nonexistence of Caustics 

Consider an area-preserving diffeomorphism T of a cylinder, that preserves the boundary 
components. 

Definition. T is called a twist map if each vertical tangent vector is turned in the same sense 
by its differential (that is, either all vectors are turned to the right, or all vectors are turned to the 
left). 

In other words, the image of every vertical line has everywhere positive (negative) deviation 
from the vertical. Let (x,y) be coordinates in the cylinder 6" 1 x R , and lift the map to the 
universal covering of the cylinder, that is to the plane with coordinates (x,y). Denote by p the 
projection on the the first factor, and let the map be f(x,y). Then the twist condition reads as 
follows: for each x the maps y — > p o f(x, y) and y — * p o / _1 (x, y) are diffeomorhisms of R 1 . It is 
also convenient to include into the definition the existence of a uniform estimate for the deviation 
from the vertical direction, that is for the derivatives d(p o f ±l {x,y))/dy. 

The billiard transformation for a smooth convex billiard acts in the cylinder with the coordi- 
nates (t, a); t is the length parameter along the billiard curve as measured from some fixed point, 
and a G [0, it] is the angle, made by the billiard trajectory with the positive direction of the billiard 
curve. The following remark is of fundamental importance. 

Lemma 1. The billiard transformation is a twist map. 

Proof. Fix a vertical line t = const in the phase cylinder. This line corresponds to a fixed 
position of the billiard ball on the billiard curve with all possible directions a allowed. Clearly, 
the greater this angle a, the greater is the arc from the fixed point to the point, at which the ball 
hits the boundary. Thus the image of the vertical line has a positive deviation from the vertical. 
Q.E.D. 

fig. 37 

We now make a digression to discuss Birkhoff's theorem on invariant curves of twist maps. 
There exist several versions of this theorem, that can be found in [Bi 2,3; He 1; Ma 1,2]. Some 
higher-dimensional generalizations are available too - [B-P]. In our exposition we follow [He 1] and 
[Ma 2]. 

Theorem 2. Let T be an orientation and area-preserving diffeomorphism of an open cylinder 
S 1 x (—1,1), that maps each topological end to itself. Let U be an open T-invariant subset, 
which contains the lower end Si x (— 1, — 1 + e) and whose closure does not intersect the upper 
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end S 1 x (1 — e, 1), e > being sufficiently small. Suppose also that the circle S 1 x {—1 + e} is a 
deformation retract ofU. Then the boundary ofU is the graph of a Lipschitz continuous function 

We will outline the idea of a proof, leaving many details aside. 

Proof. To fix ideas, assume that T is tilted to the right, that is, T turns vertical directions 
to the ones with a positive first component. Fix a circle S = S 1 x {—1 + e}, contained in U. Let 
J7+ be the set of positively accessible points, i.e., points x G U such that there exists an imbedded 
curve in U, that starts at a point of S, ends at x and has a positive deviation from the vertical. 
Similarly one defines the set of negatively accessible points U- . Let V = U+ D U- . Then V is the 
set of points, vertically accessible from S. 

fig. 38 

Notice that U± are open. Because T is a positive twist map, it preserves the property of 
an imbedded curve to have a positive deviation from the vertical. Hence T(U + n U) C U+, and, 
likewise, T _1 (?7_ n U) C E7_. Since T is area-preserving, there exists T- invariant subsets of full 
measure in U + and U- (take n, T l (U + )). The same property is enjoyed by V. 

Suppose that the invariant curve (the boundary of U) is not a graph. Then V has vertical 
segments on its boundary. Consider such a segment and suppose that V locally lies to the left 
of it. The image of this segment is a curve with a positive deviation from the vertical, and the 
image of V lies above it. Pick a point x of the T-invariant subset of V sufficiently close to the 
vertical segment; then T{x) belongs to V, and therefore is vertically accessible from S. This is a 
contradiction. If V lies to the right of the vertical segment, the same argument works with T being 
replaced by T _1 . 

fig. 39 

Let us indicate an alternative approach: one shows that U + = Ui>oT l V and U- = Ui<o T l V\ 
since T is area-preserving, it follows that U- — V and U+ — V are empty (see [LeC]). 

To see that the invariant curve is the graph of a Lipschitz function, use the following trick. 
Given a C^-differentiable function / on (—1,1), consider the diffeomorphism of the cylinder: 

F f (x,y) = (x + f(y), y); xeS 1 , y € (-1,1). 

Let Tf = Ff o T o FJ . Then FfU is invariant under Tf. If / is sufficiently close to zero in the 
Whitney C 1 topology, then Tf is also a twist map. Hence the boundary of FfU is also the graph 
of a continuous function. This holds for all functions / in a small neighbourhood of zero; hence 
the boundary of U satisfies the Lipschitz condition. Q.E.D. 

As an application of Birkhoff's theorem consider results by J. Mather on noexistence of in- 
variant curves of a billiard transformation ([Ma 1]). These results are in sharp contrast with those 
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of Lazutkin. 

First, we formulate the twist property of the billiard map analytically. As before, H(t,t') 
denotes the Euclidean distance between two points on the billiard curve whose length parameters 
are t and t' . The following statement simply means that the curve is convex. 



Lemma 3. 

d 2 H(t,t') 



> 



dt dt' 

Proof. Consider the segment tt' of a billiard trajectory, and let a and a' be the corresponding 
angle variables. Then 9H g t ' t ^ = — cos a (Section 1.2). Differentiate with respect to t': 

d 2 H(t,t') . Oa 

Since sin a > 0, one only needs to show that |p- > 0. 

Consider the twist property. The billiard transformation (t, a) — > (t', a') turns vertical vectors 
to the right: |g > 0. Hence |p > 0. Q.E.D. 

fig. 40 

We now proceed to Mather's results. 

Theorem 4. If the curvature of a C 2 smooth convex billiard curve vanishes at some point, then 
the billiard transformation has no invariant circles. 

Proof. Let t_i, to, t\ be the first coordinates of three consecutive points of a billiard trajectory. 
One of them can be viewed as a function of the other two. 

Let to be the coordinate of a point with zero curvature, and consider a two-link trajectory 
that reflect at this point. A straightforward computation, which we omit, shows that 

d 2 H(i_ 1 ,i ) d 2 H(i ,i 1 ) 

— 2 2 

dt dt 

(the reason is that the curvature vanishes at i : for the sake of the computation, one replaces the 
curve by its tangent line at to, which can be done because the curve has second order contact with 
the tangent at this point). 

One knows from Section 1.2 that three consecutive points are related by the equation: 

dH(t- U t ) | dH{tpM) _ Q 
dto dto 

Considering t as a function of t_i and ti, implicit differentiation yields: 

dto _ d 2 H(t_ u to) / ( d 2 H{t_ 1 ,to) d 2 H{toM)\ 



dt-! dt-!dt I \ dto 2 dt t 2 
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dt a 2 #(wi) H a 2 if(t_ 1; to) 9 2 if(t ,ti) 



(since the invariant circle is the graph of a Lipshitz function the derivatives exist almost every- 
where). 

In view of the above inequality, the denominators are positive whenever {t-i,t\) is sufficiently 
close to (t _ i , 1 1 ) . By Lemma 3 the numerators are positive as well. Thus 

dt ° <0, ?^<o 



for (t_i,ti) sufficiently close to (t _ i , ti ) . 

Assume now that an invariant curve exists. By Birkhoff 's theorem it is a graph of a continuous 
function. If the three consecutive points whose first coordinates are t_i,io,ii, are taken on this 
curve, it follows that to increases as a function of as well as of ii. If, in addition, t is 
sufficiently close to to, then the above inequalities for the partial derivatives hold, and they imply 
that t decreases as a function of t-\, as well as of t\. This is a contradiction. Q.E.D. 

It follows that a convex billiard with vanishing curvature at some point does not have caustics 
inside it. This statement can be proved more easily: the mirror formula from Section 2.4, applied 
to a point with zero curvature, implies that the sum of tangent segments from this point to a 
caustic is zero, which is absurd. This observation is due to M. Wojtkowski ([Wo 2]). 

Mather's theorem has a somewhat paradoxical consequence. Call a billiard trajectory e- 
glancing if, for some bounce, the angle of reflection is smaller than e. One distinguishes positive 
and negative e-glancing, according to whether the angle with the positive or negative direction of 
the billiard curve is taken into account. 

Corollary 5. If the curvature of a smooth convex billiard curve vanishes at some point then, for 
every e > 0, there exists a trajectory that is both positively and negatively e-glancing. 

Bg. 41 

Proof. Let V be the phase cylinder S 1 x [— tt, tt] of the billiard transformation T. Given a 
positive e, consider the e-strips around the boundary circles: 

V- = S 1 x (-7r,-7r + e), V+ = S 1 x (tt- e,7r), 

and set: 

W = T n {V-) U (S 1 x {-tt}). 

Assume that the statement is false. Then for some e the intersection of W and V + is void. 
Consider the connected component of V — W, that contains the upper boundary circle of the 
cylinder, and let U be the complement of this component. Then U is an invariant domain of 
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the billiard transformation, satisfying the assumptions of Birkhoff 's theorem. Its boundary is an 
invariant circle, which contradicts Theorem 4. Q.E.D. 

In fact, even a stronger result holds: under the same assumptions there exists a billiard 
trajectory, such the angle of reflection in the oriented billiard curve tends to tt, as "time" n — > +00, 
and tends to 0, as n — ► —00. We refer to [Ma 3] for strong results on chaotic dynamics in the 
so-called instability zones of twist maps. 

The above results on nonexistence of caustics were strengthened in a recent preprint by E. 
Gutkin and A. Katok [Gu-K 1]. For example, the following estimate holds. Denote by A,L,D the 
area, the perimeter length and the diameter of a convex billiard table, and let k m i n and k max be 
the minimal and the maximal values of its curvature. 

Theorem 6. If \f2D 2 k m i n k max < 1, then the billiard table contains a convex region, free of 
convex caustics, whose area is not less than A — \/2k m i n LD 2 . 

Let us also mention the results by A. Hubacher ([Flu]): if the curvature of a convex piecewise 
C 2 -smooth billiard curve is discontinuous at some point, then the billiard does not have caustics 
in a neighbourhood of the boundary. 

2.10 Aubry-Mather Theory 

Numerous results, concerning smooth convex billiards (periodic orbits, existence and nonexis- 
tence of invariant curves), hold in a more general situation, namely, for area-preserving twist maps. 
In this section we discuss some of the developments in this field, known as the Aubry-Mather theory. 
We refer to [LeC, Mo 5, Che, Ban, Ka 1] for surveys of this topic. 

Let T be an area-preserving positive twist C 1 diffeomorphism of an annulus S 1 x I with the 
area form dx A dy. Let T(x\, y\) = (X2, 2/2)- Then y2dx 2 — y\dx\ = dH, the function H being a C 2 
smooth generating function of the map T. Fix x\ and vary y\. The twist property implies that y\ 
is a function of x 2 . Hence the generating function can be considered as a function of x\ and x 2 , 
and 

dH dH 



The twist condition reads: 



2/1 = , yi = 

OX\ OX2 



P>0, or 9^X2) <Q 
oyi OX1OX2 



For convex billiards, x is the length coordinate and H is (negative) the distance between the 
corresponding points. 

To proceed further, recall some facts about orientation preserving homeomorphisms of a cir- 
cle, that go back to Poincare and Denjoy (see, e.g., [Ni, He 2] ). Let / : S 1 — > S 1 be such a 
homeomorphism (S 1 = R/Z), and let / : R 1 — ► R 1 be a lifting. Then there exists a real number 
p, well defined modulo integers, such that 

-1 < f n (x) - x - np < 1 
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for all real x and integer n. The class of p modulo integers is independent of the choice of the 
lifting and this p is called the rotation number of the homeomorphism /. 

If p = p/q is rational, p and q > being coprime, there exists x 6 R such that f q (x) = x + p. 
The projection of this x to the circle has a (/-periodic orbit under /, referred to as a (p, q)-type 
orbit (once the lifting is chosen). If p is irrational and / is twice continuously differentiable, then 
/ is conjugate to a rotation by a homeomorphism (the Denjoy theorem), and every orbit is dense 
in the circle. However, there exists a C 1 smooth map / none of whose orbits are dense; the orbits 
have a common limit set which is an invariant Cantor set in S 1 (Denjoy minimal set). 

Now we can formulate a result on periodic orbits of area-preserving twist maps. Let a < [3 
be the rotation numbers of the restriction of the map T to the lower and upper boundary circles 
of the cylinder. 

Theorem 1. For any rational number p/q G [a, /3] (p and q coprime, q > 0) there exist two 
(p,q)-type periodic orbits of T, such that the first coordinates of the points of these orbits (i.e., 
points in S 1 ) are pairwise distinct. Moreover, the orbits have the following monotonicity property: 
if z\ and z 2 are liftings of two points from the union of these two orbits to the universal covering 
R x /, and z 2 lies to the right of Z\, then T(z 2 ) lies to the right ofT(z 1 ) (T being a lifting ofT). 

Specializing to billiards, one recovers two star-shaped g-periodic trajectories with rotation 
number p. The proof makes use of the variational principle: the orbits in question are critical 
points of the " length" functional 

H(x ,x 1 ) + H(x 1 ,x 2 ) + ••• + H(x q - 1 ,x q ), 

defined on sequences of real numbers (xi), i 6 Z with x i+g = Xi+p. 

Assume that a twist map is close to an integrable map (x, y) — > (x + 4>(y),y) near the boundary 
circles of the annulus. By KAM theory the map has invariant curves near the boundary. These 
invariant curves are graphs of continuous functions. Parametrize an invariant curve by two con- 
tinuous functions: x = u(t), y = v(t), where u is increasing, and u(t) — t and v(t) are 1-periodic. 
The dynamics on an invariant circle, provided by the KAM theory, is conjugate to the rotation 
through an irrational number badly approximated by the rationals. Let p be such a number for 
the invariant curve under consideration. Then T {u{t),v{t)) = [u{t + p),v{t + p)). One can exclude 
the second function v, using the partial derivatives of the generating function: 

ffi(tt(t), «(* + p)) + H 2 (u(t - p),u(t)) = 0, 

where the indices denote the partial derivatives with respect to the first and the second variables. 
Solutions to this equation, that is increasing continuous functions u(t), with u{t) — t being 1- 
periodic, determine invariant curves with rotation number p. 

Dropping the continuity requirement on u(t), one arrives to the notion of the Mather set with 
the rotation number p, given parametrically by x = u(t),y = —Hi(u(t),u(t + p)) where u(t) is a 
monotone function, satisfying the above equation, and such that u(t + 1) = u(t) + 1. 
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Theorem 2. For any real number p 6 [a, [3] , there exists an invariant Mather set with the rotation 
number p. This set lies on the graph of a Lipschitz continuous function. 

Mather's original proof is based on the study of critical points of the functional / H{u{t) , u(t + 
p)) dt, whose Euler equation defines the Mather set. 

Mather sets generalize both the periodic trajectories from the previous theorem (p = p/q; u is 
piecewise constant) and invariant curves (u is continuous). If p is irrational, u may be discontinuous. 
In this case the Mather set is a Cantor set lying on a graph of a Lipschitz continuous function, 
whose gaps correspond to the discontinuities of u. The graph itself is not uniquely determined and, 
by no means an invariant curve. These Cantor sets are in a sense the remnants of the invariant 
curves. 

The Aubry-Mather theory explains how the invariant curves provided by KAM theory disinte- 
grate as the parameter e of a perturbation of an integrable map increases. Fixing a rotation number 
badly appriximated by the rationals, one considers the corresponding Mather invariant sets. For 
small values of e they are invariant curves, but for greater e they may become Cantor sets. Let 
e* be the greatest value of e, such that for all e < e* the invariant sets are curves. Then to the 
value e* of the perturbation parameter corresponds an invariant curve which disintegrates under 
an arbitrary small increase of the parameter. This curve is necessarily not too smooth, because, 
otherwise, by KAM theory it would "survive" an increase in e. Thus invariant curves break up 
through loss of smoothness. 

We mention in conclusion, that besides an interest in the area-preserving twist maps per se, 
a motivation for the peresent theory came from the solid state physics, namely from the study by 
Aubry of the Frenkel-Kontorova model. Moreover, similar results in differential geometry go back 
to G. Hedlund , who studied minimal geodesies in two-dimensional tori, that is, the geodesies, that 
globally minimize the distance between any pair of their points. 

2.11 Miscellanea 

We collect some more results on smooth convex billiards in this section. 

First, following E. Gutkin ([Gu 1]), consider a billiard map, which admits a horizontal circle 
a = const as an invariant curve (in the usual coordinates in the phase cylinder) . The corresponding 
trajectories of the billiard ball make a constant angle with the billiard curve. An obvious example 
of such a situation is provided by a disc. A more interesting example is a billiard curve of constant 
width. For any direction such a curve has a chord in this direction, that is perpendicular to the 
curve at both end-points. This chord is a 2-periodic trajectory, and their totality is the invariant 
curve a = tt/2. What are other possible values of the constant angle a! 

fig- 42 

Theorem 1. There exists a non-circular billiard table with an invariant curve a = const, < 
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a < tt/2, if and only if this constant angle a satisfies the equation tan na = ntana for some 
integer n > 1 . 

We outline the proof, leaving aside some computations. 

Proof. Parametrize the billiard curve by the angle <j> it makes with a fixed direction. Let r{4>) 
be the radius of curvature of the curve. If (x (</)), y(<j))) are the Cartesian coordinates of a point on 
the curve, then 

x'{4>) — r ( ( t ) ) cos 0) y'(4>) — r {4 ) ) srn 0- 

fig. 43 

Consider a segment of the trajectory, corresponding to an orbit of the billiard map on its 
horizontal invariant circle. It makes the angle a with the billiard curve. Hence 

sin (f> (x((f) + a) — x(4> — a)) = cos 4> (y(4> + a) — y{<ft — a)). 
Differentiate twice and make the necessary substitutions to arrive at: 

cos a (r(0 + a) — r{4> — a)) = sin a (r'(cf) + a) + r'(<j> — a)). 
This equation is equivalent to the existence of the horizontal invariant circle at height a. Let 

oo 

r((p) = r + ^ a n cos ncft + b n sin n<p 
l 

be the Fourier decomposition of the radius of curvature. The previous equation implies: 
a n (sin na cos a — n cos na sin a) = b n (sin na cos a — n cos na sin a) = 

for all n. 

If the billiard curve is not a circle, then, for some n, a n ^ or b n ^ 0. Hence 

sin na cosa = n cos na sin a. 

This is the desired equation. Q.E.D. 

The second result we want to discuss here concerns the following problem: how big can the 
set of n-periodic points of a billiard transformation be? In particular, can a billiard transformation 
have an (non-void) open set of periodic points? The motivation for this question comes from the 
theory of the Laplace operator (Section 2.7). 

It is easy to answer this question in the real analytic category: the measure of the set of 
periodic orbits is zero. In the smooth case, the answer is readily found for n = 2: a 2-periodic 
trajectory is perpendicular to the billiard curve, and therefore lies on the horizontal circle a = tt/2. 
However, the problem becomes quite hard for greater values of n. The case of n = 3 was settled 
by M. Rychlik ([Ry]); his proof involved symbolic computations. Later the proof was drastically 
simplified by L. Stojanov ([Sto 1]) and M. Wojtkowski ([Wo 1]). We follow the later paper. 
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Theorem 2. The set of 3-per iodic points of the billiard transformation of a smooth convex plane 
billiard is nowhere dense. 



Proof. Assume that there is an open subset in the phase space that consists of 3-periodic 
trajectories. Then a 3-periodic trajectory can be included into a 1-parameter family of 3-periodic 
trajectories with one reflection point being fixed. 

fig. 44 

Denote the curvature at Xj by Ki, and set 

sin OLi 

Let Hi be the length of the corresponding trajectory segment. Consider two infinitely close rays 
XiX 2 and X[X' 2 . Let a\ and a 2 be the (signed) distances from their intersection point Y to the 
points Xi and X 2 (so, if Y lies to the left of X 2 on the line XiX 2 , then a 2 < 0, etc). Then 
ai + a 2 = H. Applying the mirror equation (Lemma 2.4.2) twice, one gets: 

111 111 

H\ a 2 C 2 ' H 2 a\ C\ 



It follows that 



H\C 2 ~ 1 H 2 C\ 



Rewrite this equation as 

{H 2 + H 3 — H 2 H S /C\) (Hi + H 3 — H1H3/C2) = H\H 2 . 

Two similar equations are obtained by a cyclic permutation of the indices. The system of three 
equations is easily solved: multiply all three, take the square root, and divide by one of them. One 
gets: 

H 1 +H 2 -H 3 = H x H 2 jC 3 = 2H x H 2 K 3 j 'sina 3 . 



Now apply the Cosine Theorem to the triangle XiX 2 X 3 . It boils down to 

Hi + H 2 — H 3 



4HiH 2 sin 2 a 3 



Hi + H 2 + H 3 

Notice that a 3-periodic trajectory is an extremum of the perimeter length functional on inscribed 
triangles. This functional is constant on its critical set: Hi + H 2 + H 3 = L for some constant L 
and all nearby 3-periodic trajectories. Compare the above two formulas to conclude: 

K 3 2 sin 2 q?3 



sin a 3 L 
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But this is clearly absurd, because one can change the angle 0:3, keeping the pont X 3l and therefore, 
the curvature K 3 , fixed. Q.E.D. 

With a little more effort one shows that the set of 3-periodic orbits has the zero measure. We 
refer to the papers of Rychlik or Wojtkowski for this refinement. 

Almost nothing has been said about higher-dimensional billiards. To make up with this 
shortcoming, consider a smooth convex billiard in 3-space that has the following property: each 
billiard trajectory belongs to a 2-plane. An example is provided by the billiard inside a ball; are 
there other billiards with this property? The answer was found by R. Sine ( [Sin] ) . 

Theorem 3. Balls are the only billiard tables with the above property. 

Proof. Let M be a billiard surface. Given a point x 6 M, denote by l(x) the normal line to 
M at x. Pick a point x on M and consider a plane P through l{x). Given a point y of the curve 
P H M, the billiard ball, shot from x to y, will stay in the plane P. It follows that l(y) C P. Let 
x' be the "antipodal" point of intersection of l(x) with M. As y goes to x', one concludes that 
l(x') C P. This holds for any plane P through l(x); thus l(x') = l(x). 

fig. 45 

Consider the point z € M such that the line l(z) is perpendicular to P. Let Q be the plane 
through l(z) and y. Arguing as above, one concludes that l(y) C Q. Hence l(y) passes through 
the point of intersection of l(z) and P - the "center" of the section of the billiard table by the 
plane P. This conclusion holds for all points y € P H M; hence this section is a circle and xx' is 
its diameter. 

Finally, revolve the plane P about l{x). Each section of M by this plane is a circle with the 
same diameter xx' . Thus the billiard surface is a sphere. Q.E.D. 

The result holds in dimensions greater than 3 as well - see [S-K]. 

Let us mention in conclusion a recent work by Gutkin and Knill [Gu-Kn]. Given a convex 
curve 5 consider the one-parameter family of billiards for which 5 is a caustic; these billiards are 
obtained by the string construction. Let I be the variable string length and 72 the corresponding 
billiard curve. The billiard transformation associated to induces a homeomorphism of 5; let 
p(l) be its rotation number. Then p(l) is a continuous nondecreasing function. One of the results 
of [Gu-Kn] reads: for a generic 5 the function p(l) is a "devil-staircase"; this means that for any 
rational q the interval p~ 1 (q) has a non-empty interior. 
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3. Billiards in Polygons 



This chapter concerns billiards in polygons and polyhedra, mostly convex. Section 1 starts 
with the method of unfolding billiard trajectories to be used throughout the chapter. The rest of the 
section concerns dynamical properties of a torus translation, to which the billiard transformation in 
a cube reduces. Section 2 deals with encoding billiard trajectories with irrational slopes in a cube 
according to the faces they reflect in. Such encoding sequences are quasi-periodic; their complexity 
is found in the two and three dimensional cases. Section 3 starts with a criterion for stability of 
periodic billiard trajectories in polygons; this means that a trajectory "survives" a perturbation 
of the billiard table. Next we describe the set of trajectories that reflect in the same sides of 
the billiard polygon; the results imply that the entropy of polygonal billiards vanishes. Section 4 
concerns rational billiard polygons: we partition the phase space into invariant surfaces and study 
the billiard flow on them. Section 5 is a brief discussion of recent strong results on rational billiards 
via the theory of quadratic differentials; we only outline the main ideas and concepts. The last 
section concerns the relation between the dynamics of point masses and billiards, the billiard in a 
polyhedral angle and periodic billiard trajectories in triangles. 
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3.1 Square Billiards 

We start the study of billiards in polygons wih the simplest case of a square. It is surprising 
how much one can actually say about it! 

First, let us describe the procedure of unfolding a billiard trajectory. Instead of reflecting 
the trajectory in a side of a polygon reflect the polygon in this side. In this way the trajectory, 
straightened to a line, piercing a number of isometric copies of the billiard polygon. 

fig. 46 

Thus a correspondence is established between the billiard trajectories in a square and straight 
lines in the plane with a square grid, which may be assumed unit (we disregard the fact that the 
billiard trajectories stay away from the vertices; the unfolding argument shows that for a square 
the trajectories can be extended through the vertices). Two lines in the plane correspond to the 
same billiard trajectory if they differ by a translation through a vector from the lattice 2Z + 2Z. 

Notice that two neighbouring squares have the opposite orientations - they are symmetric 
with respect to their common side. Consider a bigger square that consists of four unit squares with 
a common vertex, and identify its opposite sides to obtain a torus. A billiard trajectory becomes 
a geodesic line on the flat torus: closed if the trajectory is periodic, and infinite dense geodesic 
otherwise. 

fig. 47 

Consider the trajectories in a fixed direction a. The billiard flow gives rise to the map / of 
the circle S 1 = R 1 /Z, shown in the figure. This map is a rotation: 

f(x) = x + cota (modi). 

The dynamical properties of the billiard flow in a fixed direction correspond to those of this rotation; 
say, if the rotation is ergodic, then so is the flow etc. Likewise the billiard dynamical system in 
an n-dimensional cube decomposes according to the directions of the trajectories; each component 
has a section which is a (n — l)-dimensional torus rotation. 
Thus one is led to the study of a rotation of the torus T n : 

T a : (xi,...,x n ) -> (xi + a t (modi), ...,x n + a n (modi)), 

equipped with the Lebesgue measure fx = dx\...dx n . 

Notice that if the trajectory of some point is dense, then any other trajectory is dense as well: 
it follows from the fact that the torus is a homogeneous space of R n , acting by translations. The 
main properties of the torus rotations are contained in the following two theorems, due to Weyl 
and to Kronecker-Weyl, respectively (see, e.g., [Si 1]). 
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Theorem 1. The rotation T a is ergodic if and only if the numbers a±, ...,a n are independent over 
the integers (that is, if r\a\ + ... + r n a n 6 Z for n, ...,r n G Z then n = ... = r n = 0). 

Proof. Let ai, ...,a n be independent over Z, and let / be a T a -invariant mod measurable 
function (which means that f(T a x) = f(x) almost everywhere). One wants to show that / is a 
constant off a set of zero measure. 

First, one may assume that / is bounded. Otherwise, consider the set S = {x : \ f\ < c}, and 
replace / by x(S)f, where x 1S the indicator of the set (1 inside, outside). This new function is 
still r a -invariant and bounded, so the result for / will follow from that for x(S)f as c — > oo. 

Let 

/(x)~^c r e 2 -<^> 

r 

be the Fourier expansion of /, where r = (ri,...,r n ) is an integer vector, and the summation is 
over all such vectors. Since / is T a -invariant, 

f(T a x) ~ J]c r e 2 W «r ) a> e 2 W i<r,x> = ^ ^ e ^i<r,x> „ (w Q y 

The Fourier coefficients are unique, hence 

2iri<r,a> _ 

for all r. If c r 7^ then < r, a > is an integer, which is impossible by the assumption. 
Conversely, if < r, a >G Z then the function e 2vi < r ^ x > i s T a -invariant. Q.E.D. 
As a matter of fact irrational rotations are uniquely ergodic. 

In an ergodic system the trajectory of almost every point spends the time in a measurable 
set asymptotically proportional to its measure; it follows that the trajectory of almost every point 
is dense. In view of the above remark, all orbits of T a are dense, provided that a\,...,a n are 
independent over the integers (this fact has an elementary proof; the reader is encouraged to find 
it). Apply this to the orbit of the "origin" (0, ... , 0). 

Theorem 2. Let the irrational numbers 01, a n be linearly independent over the rationals. Then 
for every positive e there exist the integers k and mi, ...,m n such that 

\k cii — m,j| < e, i = l,...,n. 

Proof. There exists a number k such that the fc-th image of the origin under T a is e- close to 
itself: k cti ~ e (mod 1). These are the desired inequalities. Q.E.D. 
As far as the entropy is concerned, the following result holds. 

Theorem 3. The metric entropy (with respect to the Lebesgue measure) of a torus rotation equals 
zero. 

Proof. This follows from two properties of entropy, mentioned in Section 1.11. A rotation of 
a torus is the product of rotations of circles, so it is enough to consider this 1-dimensional case. If 
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the angle of a rotation T is 7r-rational, then T n = id for some n. Since h(T n ) = nh(T), the entropy 
of T vanishes. Otherwise the partition of the circle in two semicircles is a one-sided generator, 
hence h(T) = 0. Q.E.D. 

Consider an irrational rotation T of the circle R 1 /Z. The orbit of each point spends the 
average time in an interval, asymptotically equal to its length (Weyl's theorem): 

lim {k : < k < n, T k (x) G 1} = /z(I). 

n— s-oo 

One says that the orbit is equidistributed in the circle. 

One cannot help mentioning an application to a problem from number theory (see [Ar 5, Ki]). 
Consider the powers of 2 : 1, 2, 4, 8, ... ; do they start more often with 7 than with 8? 

2 k starts with 7 if and only if 7 • 10 m < 2 k < 8 • 10 m for some integer m. Take the logarithm 
with the base 10: 

lg 7 + m < k lg 2 < lg 8 + m. 

Consider the rotation T of the circle R^/Z through lg2. The average number of times 2 k starts 
with 7 is the number of times T fc (0) 6 [Ig7,lg8). This number equals lg(8/7). Likewise one deals 
with the first digit of 8. Thus the powers of 2 start with 7 more often than with 8, namely, 

lg(8/7) 
lg(9/8) 

times as often. 

Return back to the billiard inside a square. Its phase space is partitioned into disjoint compo- 
nents, corresponding to different directions a of the trajectories. If tana is rational, all trajectories 
are periodic; otherwise the billiard flow is ergodic in this component and each trajectory is dense 
in the configuration space (that is, in the square). This dichotomy - a trajectory is either periodic 
or dense - does not hold for a general polygon, as we will see later. 

To conclude this section we analyse periodic trajectories of the billiard in the square. The 
unfolding of such a trajectory is a segment in the plane whose end-points differ by a translation 
through a vector from the lattice 2Z + 2Z. A nearby parallel trajectory is also periodic with the 
same number of links and the same length. Thus periodic trajectories come in parallel beams. 

fig. 48 

To describe the length spectrum, assume that an unfolded trajectory goes from the origin to 
the lattice point (2p,2q). A trajectory in the south-east direction will go to the north-east after 
a reflection, so, without loss of generality, we assume that p and q are nonnegative. The length 
of the trajectory equals 2^Jp 2 + q 2 , and to a choice of p and q two orientations of the trajectory 
correspond. Hence the number of periodic trajectories of length less than L is the number of 
nonnegative integers, satisfying the inequality p 2 + q 2 < L 2 /2. 



56 



In the first approximation, this number is the number of integer points inside the quarter 
of the circle of radius L/y/2. Modulo terms of lower order, it equals the area, that is, ttL 2 /8. 
Hence the number of periodic families of length less than L satisfies N(L) ~ ttL 2 /8. The error 
N(L) — 7rL 2 /8 is of the order of the circumference length, i.e.N(L) = irL 2 /8 + O(L) (see [Be 4]). 

We mention in conclusion that billiards in rectangles, equilateral triangles and the right trian- 
gles with an acute angle tt/4 or 7r/6 are investigated in a similar way. Reflecting any one of these 
polygons in their sides, one generates a plane lattice; thus the billiard flow reduces to a linear flow 
on a torus. 

3.2 Symbolic Description of Billiards in Squares and Cubes 

Unfolding a nonperiodic billiard trajectory in a square yields a line L in the plane with an 
irrational slope A. The plane is equipped with the unit grid, and, travelling along the line, one 
meets the vertical and the horizontal segments of the grid (we assume the line avoids the vertices; 
in any case it can happen at most once). Encode the former event by 1 and the latter by 0. Thus 
to the line L an infinite sequence £ of 0's and l's corresponds, which we call the cutting sequence 
of the line. This sequence determines in which sides of the square - vertical or horizontal - occur 
consecutive reflections of the billiard ball. 

fig. 49 

Assume that A > 1 and let [re ,ni,n2, ...] be the continuous fraction expansion of A: 

A = n H 3_ . 



n 2 H ; 

n 3 + ... 

In particular, no = [A]. The line L never intersects two consecutive vertical segments, so the cutting 
sequence consists of blocks of 0's, separated by isolated l's. How many 0's can there be in a block? 
By comparizon of the slope of L with those of the two broken lines in the figure one concludes that 
the number of 0's in a block equals either no or no + 1. Following [Se], call the minimum of the 
two the value of the sequence . Likewise, the number N of 0's in the segment 10. ..010. ..01. ..10. ..01 
(k blocks) satisfies the inequalities kX — 1 < N < kX + 1, and, therefore, may assume only two 
different values. If the slope A < 1, the roles of 0's and l's are reversed, and A gets replaced by 
1/A in the previous formulas. 

fig. 50 

The cutting sequence £ can be encoded in the new symbols: 0' = 0, 1' = 0...01 (n zeroes). 
For example, if the value of the sequence equals 1, then 

...0101001001... = ...l'l'O'l'O'l'... 
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Call this new sequence £' the predecessor of £. It might happen so that £' also has a predecessor 
etc. The following observation was made more than a century ago by Christoffel and H.J. S.Smith. 

Theorem 1. The cutting sequence £ has predecessors of all generations, and their values are 
ni,n 2 , ... 

Proof. Consider another grid in the plane - the image of the original one under the linear 
transformation 



Encode the sides of the parallelograms of the new grid by the symbols 0' and 1', according to 
whether their preimages are horizontal or vertical. Then the pedecessor £' is the cutting sequence 
of the line L with respect to the new grid. This is the same as the cutting sequence of the line 
A~ X (L) with respect to the original square grid. The slope of A _1 (L) equals A — n < 1. Thus the 



As an instructive example, consider the line L through the origin whose slope is the golden 
ratio 1+ 2 V ^ . Its cutting sequence £ enjoys a remarkable property. 

Theorem 2. £ is invariant under the substitution — > 01, 1 — > (so one creates the sequence 
starting with and iterating the substitution). 

Proof. Consider the linear transformation 



It preserves the line L and sends the integer lattice to itself. Hence the cutting sequence r\ with 
respect to the new grid is the same as the old one £. The figure depicts that in £ corresponds to 
01 in r], and 1 in £ - to in rj. Q.E.D. 



Leaving aside other substitution rules, corresponding to the lines whose slopes are quadratic 
irrationalities, and the relation to the theory of quasicrystals (see, e.g., [Br, S-T]), we proceed to 
quasiperiodicity of the cutting sequences of lines with irrational slopes. 

Theorem 3. The cutting sequence is not periodic; however, each finite segment appears in it 
infinitely many times. 

Proof. If the positions of two billiard balls in a square are close enough and they go in 
the same direction, then they make sufficiently many consecutive reflections in the same sides 
of the square. One knows from the previous section that a billiard trajectory with an irrational 




value of the new cutting sequence is [ ] . Q.E.D. 



fig- 51 




fig. 52 
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slope will come infinitely many times to a neighbourhood of any point in the same direction. The 
quasiperiodicity follows. Since the slope of the line equals the average number of O's per block of 
the cutting sequence, periodicity of the sequence implies rationality of the slope. Q.E.D. 

The next property of the cutting sequences was discovered by M. Morse and G. Hedlund in 
their study of symbolic dynamics ( [M-H] ) . 

Definition. The complexity function p(n) of an infinite sequence in some symbols is the 
number of its distinct n-element segments. 

The following result says that the complexity of the cutting sequences of the lines with irra- 
tional slopes is the minimal complexity of nonperiodic sequences in two symbols (such sequences 
are called Sturmian sequences). 

Theorem 4. p(n) = n + 1. 

Proof. Since a billiard trajectory with an irrational slope comes arbitrarily close to any point 
of the square, the sets of finite segments of the cutting sequences of any two parallel trajectories 
coincide. Thus one can find complexity by computing the number of different initial segments of 
length n in the cutting sequences of the lines with a given slope . Partition the square grid into 
"ladders', as shown in the figure. The fc-th symbol in the cutting sequence is or 1, according to 
whether the line L meets a horizontal or a vertical segment of the fc-th ladder. 

fig. 53 

Let ei and e2 be the unit orthogonal frame in the plane. Project the plane onto the diagonal 
x + y = along L, and factorize the diagonal by the translation through e\ — e^- Identify the 
quotient space with the unit circle S; the projections of the vertices of the first ladder partition S 
into two irrational arcs. Let T be the rotation of the circle through the length of an arc, that is, 
through the projection of e\. 

The number of different initial n-segments of the cutting segments of the lines with the given 
slope equals the number of segments into which the projections of the vertices of the first n ladders 
partition the circle S. Each ladder is obtained from the first one by translation through e\. Hence 
the complexity equals the number of points in the orbit T l (0), i = 0, n. Since T is an irrational 
rotation, all points of the orbit are distinct; thus p{n) = n + 1. Q.E.D. 

The theorem was recently generalized to the billiard in a 3-dimensional cube - see [A-M-S- 
T]. Consider a line in 3-space in a totally irrational direction (this means that the components 
of the direction vector are rationally independent). In space the "grid" of the planes is given: 
Xi = n; i = 1,2,3; n £ Z. One defines the cutting sequence in three symbols, according to the 
direction of the consecutive planes the line meets. 

Theorem 5. The complexity of the cutting sequence equals n 2 + n + 1. 
fig. 54 
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The proof proceeds along the lines of the previous one. Instead of the circle S one gets a 
2-torus, represented by a hexagon (the projection of a unit cube along the line onto the diagonal 
plane x + y + z = 0), whose opposite parallel sides are pasted together. The projection of the first 
ladder (which is now a polyhedral surface) partitions the torus into three parallelograms. Also 
an irrational rotation T of the torus is given (through the projection of a unit basic vector onto 
the diagonal plane); and the problem consists in computing the number of pieces in the partition 
of the torus, obtained by the intersection of the first n iterations of the initial partition under T. 
The numbers of vertices and edges of this intersection are found to be n 2 + 2 and 2re 2 + n + 3, 
respectively. Since the Euler characteristics vanishes, one finds the number of faces to be n 2 + n + 1. 

See Theorem 3.4.4 for another generalization of Theorem 4. 

3.3 Unfolding Trajectories in General Polygons 

A billiard trajectory in a general polygon P can be unfolded exactly in the same way as we 
did in Section 3.1. Each successive copy of the billiard polygon is obtained from the previous one 
by reflection in the side, met by the straightened trajectory. 

fig. 55 

Consider the group of motions of the plane G(P), generated by the reflections in the sides of 
P. Denote the reflection in side s by a s . Notice that cr CTs ( t ) = UsatcrJ 1 for any two sides s and t. 
It follows that every copy of P, involved in the unfolding, is the image of P under a motion from 
the group G(P). Notice that the product of an even number of generators of the group preserves 
the orientation of the plane, while an odd number reverses it. 

To study the directions of the trajectories one projects the group of motions of the plane onto 
the orthogonal group 0(2, R). Denote the image of G(P) in the orthogonal group by 0(P). This 
group is generated by the reflections in the lines through the origin, parallel to the sides of the 
polygon. The product of two reflections is the rotation through the angle twice the angle between 
the corresponding lines. 

Definition. A polygon P is called rational if 0(P) is a finite group and irrational otherwise. 

Thus the angles of a rational polygon are rational multiples of n; if a polygon is simply 
connected this condition is also sufficient for being rational. A finite subgroup of 0(2), generated 
by reflections, is the group of symmetries of a regular polygon, called the dihedral group. Since 
the direction of a billiard trajectory changes by an element of the group 0(P), when it hits the 
boundary, the number of directions of a given trajectory in a rational polygon is finite. 

Consider an re-periodic billiard trajectory and unfold it to a line I. The motion g from the 
group G(P), that sends P to its n-th copy along I, preserves this line. One of two things hold: 
either g is orientation reversing - then n is odd and g is a glide reflection in /, or g is orientation 
preserving - then re is even and g is a parallel translation in the direction of I (we refer to [Be 1] 
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for these facts from plane geometry). In the latter case the trajectory is included in a strip, that 
consists of a one-parameter family of periodic trajectories with the same periods and lengths. In 
the former the trajectory is also included in a one-parameter family of periodic trajectories, but 
now their periods and lengths are twice as big. 

fig. 56 

Definition. A periodic billiard trajectory in a polygon is called stable if an arbitrary small 
perturbation of the vertices leads to a perturbation of the trajectory, but not to its destruction. 
For example, a 2-periodic trajectory in a rectangle is not stable. 

fig. 57 

Enumerate the sides of the polygon by 1, 2, k clockwise. Consider an even-linked periodic 
trajectory and let i\, im be the numbers of the sides, visited successively. The following criterion 
of stability was found in [G-S-V]. 

Lemma 1. The trajectory is stable if and only if the numbers in the list i\, ■■■,i2n can be par- 
titioned in pairs of equal numbers, so that the number from each pair appears once at an even 
position, and once at an odd one. 

Proof. Let be the angle made by the i-th side with a fixed direction. The 2n-th copy of 
the polygon along the unfolded trajectory is parallel to it. The composition of two reflections is a 
rotation, and the angle, through which the consecutive reflections in the sides i±, ■■■,i2n rotate the 
plane, equals 

Hence this angle is a multiple of 2ir, and, for the trajectory to be stable, the variation of the angle 
under a perturbation of the polygon should satisfy: 

da it - Sa i2 + ... + Sa i2n _ 1 - 5a i2n = 0. 

However, an arbitrary variation of the vertices induces an arbitrary variation of the directions of 
its sides. Hence the previous relation holds if and only if the terms in it cancel pairwise. Q.E.D. 

Corollary 2. If the angles of a polygon are independent over the rational numbers then every 
periodic billiard trajectory in it is stable. 

Proof. In the previous notation, 

OLi x - a i2 + ... + a i2n _ 1 - a i2n = (mod2Tr). 

If the indices i\, i2 n do not satisfy the previous criterion, one obtains a nontrivial rational linear 
relation on the angles a\, 7r, which implies a rational dependence of the interior angles of 
the polygon. Q.E.D. 
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For instance, a generic convex 2ra-gon does not admit a 2n-link nonintersecting billiard tra- 
jectory. A necessary condition for such a trajectory to exist is that the alternating sum of the 
interior angles vanishes (see [Be 1]). We refer to [G-S-V] for other results and examples concerning 
stability of billiard trajectories in polygons. 

We proceed to some results from [G-K-T] that strengthen the ones from [B-K-M] . assume that 
the billiard polygon is simply connected. As before, the sides are assigned the numbers 1, k, and 
an orbit gets encoded by the sequence of the numbers of the sides it successively hits. Consider 
the part of the phase space of the billiard transformation T, that consists of the positions of the 
billiard ball on the boundary and its inward unit velocities such that the forward orbit of the ball 
never hits a vertex. Abusing the notation, call it V. For x € V denote by w(x) the sequence that 
encodes the forward orbit of x. Given a sequence w denote by X(w) the set of phase points whose 
forward orbits are encoded by w (in particular, their foot points belong to the same side of the 
polygon). Call a set S C X(w) a strip if all x 6 S are parallel vectors whose foot points constitute 
an interval. If the foot points form an open interval, call it an open strip. The image of a strip 
under the billiard transformation is again a strip of the same width. 

fig. 58 

We start with two remarks. First, if the directions of two vectors x and y from V are not 
parallel, then w(x) ^ w(y). Indeed, their unfolded trajectories linearly diverge, so some vertex of 
the copies of P will fall into the angle between them. The first time it happens the reflections of 
the two trajectories occur in different sides. 

fig. 59 

Secondly, given a word w and two parallel vectors x and y in X(w), any parallel vector, whose 
foot point lies between those of x and y, also belongs to X{w). Hence X{w) is the maximal strip, 
corresponding to the word w. The trajectories of its boundary points come arbitrarily close to 
some vertices of the polygon. We mention also, that the maximal strip, that contains a vector x, 
is sent to the maximal strip for T{x) by the billiard transformation T. 

Theorem 3. Let w be an n-periodic word. Then each vector from X(w) has a periodic trajectory 
(of period n if n is even; if n is odd then one element of X(w) has an n-periodic trajectory and 
the rest - 2n-periodic ones). The set X(w) is an open strip. 

Proof. Assume n is even (otherwise, double it). Pick x € X(w) and unfold its trajectory. 
We claim that the n-th copy of P along the trajectory is parallel to P. Indeed, otherwise x and 
T n (x) have the foot points on the same side of the polygon (because the word w is n-periodic), 
but their directions are not parallel. Hence w(x) ^ w(T n (x)), which contradicts the periodicity of 
w. Thus the n-th copy of P is obtained from P by a parallel translation in the direction of x. This 
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means that x is a fixed point of T n , so its trajectory is periodic. Since a periodic trajectory stays 
a bounded distance away from the vertices, the strip X(w) is open. Q.E.D. 
Unlike periodic trajectories, non-periodic ones are never included in strips. 

Theorem 4. For a non-periodic sequence w the set X(w) consists of at most one point. 

Proof. If there are two points in X(w) then X(w) is a nonzero width strip. Let x £ X(w) 
have its foot point on the strip's median. Consider the forward limit set Y of x under the billiard 
transformation T. Since the width of X{w) is positive, the trajectories of the points from Y stay a 
bounded distance away from the vertices. Hence Y C V, and the restriction of T to Y is continuous. 
Being bounded and closed, Y is compact. 

Use the strengthened version of the recurrence theorem from Section 1.6 due to H. Fursten- 
berg ([Fu]) to conclude that there exists a uniformly recurrent point y G Y (it means: for any 
neighbourhood W of y there exists a constant C > such that the return times n« > 0, defined by 
T ni {y) 6 W, satisfy: n i+1 — rij < C). Pick a positive e and consider the unfolded maximal strip S 
for y together with its e-neighbourhood S e . 

fig. 60 

According to the remark, preceding Theorem 3, some vertices of the copies of P, unfolded 
along the strip S, fall into S e . We claim that they fall with uniformly bounded gaps into each of 
the two components of S e — S. Consider the leftmost point y' of S. Find m such that the foot 
point of T m {y') is within e/2 of a vertex of the polygon, and assume it to be the first such vertex 
along the strip. Consider the neighbourhood W of y such that for any z € W the m-th iterate 
T m (z) is e/2-close to T m (y). Let z' be the vector, parallel to z, whose foot point concides with 
that of y'. Then T m (z') is e/2-close to T m (y'). Let be the recurrence times for y to W. Then 
m + n-i are the recurrence times for y' to the e-neighbourhood of the vertex under consideration, 
and this sequence has uniformly bounded gaps. 

Recall that y is a forward limit of x: there is a sequence nj — > oo such that Xi = T ni (x) — > y. 
No two vectors Xi and Xj may be parallel. Indeed, they do not coincide because the trajectory of x 
is not periodic. Also both distances from the foot point of x to the two boundaries of its unfolded 
maximal strip can only increase under T. Hence if Xi is parallel to Xj, i < j, then the maximal 
strip for Xj contains at least one boundary of the maximal strip for Xi in its interior. This would 
imply that there are some vertices inside the unfolded strip for xj, which is impossible. 

fig. 61 

Finally, consider the intersections of the unfolded strips for Xi with S e — S. As i — > oo, the 
diameter of this intersection goes to infinity. Because the gaps between the vertices that fall into 
S e — S are uniformly bounded, a vertex will eventually appear in the interior of the unfolded strip 
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for Xi. This is a contradiction. Q.E.D. 

It follows that the closure of a non-periodic billiard trajectory in a polygon contains at least 
one vertex. Previously this was proved in [B-K-M] for every position of the billiard ball and almost 
every direction at it. Similar results hold for the geodesic flow on a polyhedral surface and for 
3-dimensional polyhedral billiards ([G-K-T]). 

An important consequence of the two previous theorems concerns the topological entropy of 
the billiard transformation T for a polygon. Topological entropy was defined for continuous maps, 
while the billiard transformation has discontinuities. The authors of [G-K-T] refer to [P-P] for an 
appropriate definition of h top in the discontinuous case. 

Corollary 5. h top (T) = 0. 

The above statement was first appeared in the paper by A. Katok [Ka 2] . It implies that the 
metric entropy vanishes as well ( proved in [B-K-M] and [Si]). 

We only outline the proof. Consider the set Y of sequences in 1, 2, k that encode the billiard 
trajectories for time from — oo to oo. Let S be the shift transformation. Then the encoding map 
E : V — > Y conjugates the billiard transformation and the shift : ET = SE. By the previous 
results, S has a one-sided generator - the partition into k parts by the value of the zero symbol in 
the sequence (i.e., the side to which the foot point of a vector belongs). Hence for a shift-invariant 
measure on Y its metric entropy vanishes. 

To apply the variational principle for topological entropy, consider the closure of Y in the space 
of sequences. A. Katok proved in [Ka 2] that every ergodic non-atomic shift-invariant measure on 
Y is supported on Y. Thus topological entropy of S on Y vanishes. To conclude that topological 
entropy of T vanishes too, one uses a refinement of a theorem by R. Bowen (see [G-K-T] for details). 

It follows that certain quantities, associated with the billiard in a polygon, grow slower than 
exponentially (subexponentially in the terminology of [Ka 2]): the number of different words of 
a fixed length in Y; the number of families of periodic trajectories with the period not greater 
than a given number; the number of generalized diagonals, that is, billiard trajectories from one 
vertex to another, etc. It was conjectured that these numbers grow at most polynomially, but it 
is not proved yet (see [Gu 2] and [Ka 2] for a discussion) . The property of having zero topological 
entropy is also proved in [Gu-H] for a broader class of transformations - the generalized polygon 
exchanges, that include polygonal billiards. 

We mention in conclusion a result by M. Boshernitzan ([Bos]). Recall that the phase space 
of the billiard is dP x S 1 . A direction v 6 S 1 is called exceptional if, for some x € dP, the 
trajectory of (x, v) is a generalized diagonal. A direction v = exp(27r</>) 6 S 1 is called rational if 
is a rational number. Boshernitzan's theorem states that there exist only finitely many directions 
that are exceptional and rational at the same time. 

3.4 Rational Polygons 

Recall that rational polygons are the polygons P such that the group C(P), introduced in the 
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previous section, is finite; for a simply connected polygon this is equivalent to the condition that 
all the angles are 7r-rational. In a rational polygon a given billiard trajectory may have only finitely 
many different directions. This makes it possible to decompose the phase space into a collection 
of smaller invariant subspaces. This decomposition first appeared in [F-K], and then had been 
rediscovered in [K-Z, Ke] and [R-B] (see [Gu 2] for a reference). 

To fix ideas let P be a simply connected rational polygon. The group 0(P) is the dihedral 
group D]\r generated by the reflections in lines through the origin that meet at angles ir/N, where 
N is a positive integer. This group has 2N elements, and the orbit of a generic point 9 ^ kn/N 
of the unit circle consists of 2N points. If the angles of the polygon are irnii/rii, where m; and 
i%i are coprime integers, then N is the least common multiple of nj's. Consider the phase space 
of the billiard flow P x S 1 , and let Rg be its subset of points whose second coordinate belongs to 
the orbit of 9 under Dm- Since a trajectory changes its direction by an element of Dn under each 
reflection, Rg is an invariant surface of the billiard flow in P. 

To construct Rg (for 9 ^ kn/N) consider 2N disjoint parallel copies of P in the plane. Call 
them Pi,..., P2N, and orient the even ones clockwise and the odd ones counterclockwise. We will 
paste their sides together pairwise, according to the action of the group Djy. Let < 9 = 9i < ir/N 
be some angle, and let 9i be its i-th. image under the action of Dn (see figure 62). Consider Pi 
and reflect the direction 9i in one of its sides. The reflected direction is 9j for some j. Paste the 
chosen side of Pi to the identical side of Pj . After these pastings are done for all the sides of all the 
polygons one obtains an oriented closed surface R. The result does not depend on the choice of 
the original angle 9, hence we suppress it from the notation. Figure 62 illustrates our construction 
for a right triangle whose acute angle equals tt/8. The surface R has genus 2 in this example. 



Consider the general case. Let the angles of the billiard fc-gon be irrrii/rii, and N be the least 
common multiple of rij's. In the process of pasting one consecutively attaches 2rii copies of the 
i-th angle of the polygon around its i-th vertex to obtain a multiple of 2ir (in the above example 
16 copies of the angle 37r/8 are needed; they sum up to 671"). The number of copies of the i-th. 
vertex before pasting was 2N, hence, after pasting, there remain N/rii copies of this vertex. Thus 
the total number of vertices in R is N ^ The total number of edges is 2kN/2 = kN, and the 
number of faces is 2N. Therefore the Euler characteristic of R equals 



fig. 62 




the last equality follows from the formula for the sum of angles of a /c-gon: 




Consequently the genus equals 
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The billiard flow on the surface R is obtained from the constant flows in the directions 9i in 
the polygons Pj. The result is a vector field on R with singularities at the vertices. The i-th vertex 
of R is the result of pasting 2n^ copies of the angle nrrii/ni, which sums up to 27rm,j. However 
the angle at the corresponding vertex of R is 2ir. Thus one geometrically realizes the pasting by 
scaling down the angles at the i-th vertex by the factor mj. The result is a multisaddle singularity, 
shown in figure 63 for mj = 3. 

fig.63 

We remark that the flow on R is free from singularities if and only if r/ij = 1 for all i. Since 
the sum of all angles equals 7r(k — 2) one gets the following equation in this case: 

— + ... + — = k-2. 

The only integer solutions > 2 are, up to a permutation: 

(1/3,1/3,1/3), (1/2,1/4,1/4), (1/2,1/3,1/6), (1/2,1/2,1/2,1/2) 

(exercise for the reader). The corresponding polygons, sometimes called integrable, were already 
mentioned in Section 3.1: an equilateral triangle, a right isosceles triangle, a right triangle with 
the acute angles 7r/6 and tt/3, and a rectangle. In each case the billiard flow reduces to a constant 
flow on a torus. 

After the surface R is constructed one can cut it along some of its edges to get a connected 
polygon Q in the plane (if its pieces happen to overlap we consider them as belonging to different 
copies of the plane). This is similar to cutting a paper cube (or a more general polyhedron) along 
its edges to flatten it. The resulting polygon Q is by no means unique. Its sides are partitioned 
into pairs of parallel equal sides that are to be pasted together pairwise to recover the surface R. 
The billiard flow in a fixed direction is realized as a constant flow, with the understanding that 
once the billiard ball hits a side it instantaneously reappears at the corresponding point of the 
parallel side to proceed in the same direction. 

fig. 64 

What we have achieved so far is a decomposition of the billiard flow in a rational polygon into 
a one-parameter family of flows Fg on the surface R. The flows in different directions are obtained 
one from another by a rotation. Recall that a flow is called minimal if any of its orbits is dense 
(see section 1.10). 

Theorem 1. For all but countably many directions 6 the Row Fg is minimal. 

Proof. Given a rational billiard polygon P consider the unfolding of the billiard trajectories 
in it. Each copy of P one obtains in this way is an image of P under the group G(P). This 
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group is countable, hence there are countably many different copies. Call a direction in the plane 
exceptional if it is the direction from a vertex of P to a vertex of one of its images under G(P). 
There are countably many exceptional directions. Let 9 be a non-exceptional direction. 

We claim that the flow Fg is minimal on R. First, a trajectory in this direction cannot be 
periodic - otherwise its unfolded maximal strip will contain vertices on the boundary, which implies 
that 9 is exceptional. Suppose that a trajectory t of Fg is not dense. Then there exists a maximal 
strip of non-zero width in the direction 9 in the above described polygon Q, which is never visited 
by the trajectory t. Since 9 is not exceptional, the boundary of this strip does not contain a 
segment through two vertices of Q. Thus each segment of this boundary is either a segment of t 
or the limit of segments of t. Since the width of the strip is positive and the area of R is finite, 
the strip must be periodic. Then t is periodic as well, which was already found to be impossible. 
Q.E.D. 

Projecting back to the billiard polygon P we see that for all but countably many directions 
every billiard trajectory is dense in P. Following [K-Z] we apply the previous considerations to 
arbitrary billiard polygons. Recall (Section 1.10) that a flow (or a map) is called topologically 
transitive if it has a dense orbit. Consider the space X of simply connected A;-gons with its obvious 
topology (the product of k copies of the plane). Recall that a G§ set is a countable intersection of 
open sets. 

Theorem 2. There is a dense Gs subset in X consisting of polygons with topologically transitive 
billiard Hows. 

Proof. Identify the phase space of the billiard flow in each polygon from X with D 2 x S 1 ; 
assume that this identification continuously depends on the polygon. Choose a countable basis Bi 
for the topology in D 2 x S 1 . Denote by X n the set of fc-gons P which have a billiard trajectory 
that visits all the images of the sets Bi,...,B n in the phase space of the billiard flow in P. Each 
X n is open, and (~\X n is a Gs set. 

To show that it is dense consider the set Y m of rational fe-gons such that the least common 
denominator of their angles is not less than m. For a polygon P 6 Y m any invariant surface R$ 
is 1/m-dense in the phase space. Hence for any n there exists m such that for any P 6 Y m the 
surface R$ intersects all the images of Bi, B n in the phase space of the billiard in P. Since Re 
has a dense trajectory for almost all 9, we conclude that Y m C X n . Finally, Y m is dense in X for 
every m, so X n is dense for every n, and so is nX n in view of Baire's theorem. Q.E.D. 

We refer to [Gu-K 2] for similar results concerning weak mixing. 

In Part 1 we reduced the dimension of the phase space of the billiard flow from 3 to 2 replacing 
the flow with a map. Likewise, in the case of rational polygons, the billiard flow on an invariant 
surface can be reduced to a one-dimensional transformation. 

Definition. Consider a partition (Ji,...,J n ) of the interval [0,1) into nonintersecting semi- 
closed intervals, enumerated from left to right, and let a = (<ti, a n ) be a permutation of n 
elements. An interval exchange transformation T : [0, 1) — > [0, 1), associated to the partition and 
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the permutation, is a transformation whose restriction to each ij is a parallel translation and such 
that the intervals T(Ii), ...,T(I n ) follow from left to right in the order a\, ...,a n . 

For example, the exchange of two intervals [0,a) and [a, 1) is identified with the rotation 
through 1 — a of the circle R 1 /Z. 

The reduction goes as follows. Choose a side Si and an initial angle 6\, and consider the 
collection of all pairs of sides and angles visited, starting from si in the direction 9\. This collection 
consists of sides si, s& together with angles 9ij, j = 1, Zj, z = 1, k, belonging to the sides 
Si. Draw Zj copies of Sj side by side, and contract the j-th copy of Sj by the factor sin^-. Then, by 
the results from Part 1 on the invariant measure of the billiard transformation, this transformation 
induces a piecewise isometry of the collection of intervals under consideration. If the intervals are 
correctly oriented this transformation is order preserving, and therefore, is an interval exchange 
transformation (see [C-F-S, Gu 2] or [B-K-M]). 

An equivalent way to construct this reduction is by way of unfolding the invariant surface in 
the plane. If Q is the corresponding polygon then its sides sre partitioned into n pairs (s^s^) of 
equal parallel segments. A constant flow has an invariant transversal measure, "the width of a 
beam". Let / = S\ U ... U s n , and define a measure in I as the length of the orthogonal projection 
of Si along the constant flow. Then the flow induces an interval exchange in /. 

The reduction of the billiard flow to an interval exchange transformation provides an alterna- 
tive approach to the study of rational billiards. For instance, Theorem 1 can be proved along these 
lines. An important result by A. Katok ([Ka 3]), obtained in this way, states that the billiard flow 
on an invariant surface is not mixing. 

Next we show that the billiard in a rational polygon always has a periodic trajectory. This 
simple argument was found by A. Stepin (see [G-C, G-Z]) and was also mentioned in [Bos]. Shoot 
the billiard ball in the direction perpendicular to a side of the polygon. By Poincare's recurrence 
theorem, the ball will return back arbitrarily close to its position and will meet the side at an angle 
arbitrarily close to ir/2. Since the number of directions of a trajectory is finite the ball will return 
to the original side in the direction perpendicular to it. After the ball bounces off it will repeat 
its motion along the same trajectory backwards, so this trajectory is periodic. A similar argument 
applies to a rational polytope in any dimension, that is a polytop with the property that the group, 
generated by the reflections in hyperspaces, parallel to its faces, is finite. 

fig. 65 

We also mention almost integrable billiards studied by E. Gutkin ([Gu 2,3]). These billiards 
occupy an intermediate position between the integrable ones and the ones in rational polygons. 
A polygon P is called almost integrable if the group G(P) is a discrete subgroup of the group of 
motions of the plane. There are four such subgroups, generated by reflections in the sides of the 
four integrable polygons. An almost integrable polygon is a polygon that can be drawn on the 
corresponding lattice. 
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fig. 66 



Choose a basis e\,e2 for this lattice; a direction a\e\ + 0262 is called rational if 01/02 ia a 
rational number (this does not depend on the choice of the basis). E. Gutkin proved that the 
following conditions are equivalent for an almost integrable polygon: a direction is irrational; the 
billiard flow on the invariant surface in this direction is minimal; this flow is ergodic; this flow 
is aperiodic (i.e., for no t its time t map is equal to identity). This result is specific for almost 
integrable polygons: it does not hold for an arbitrary rational polygon. 

In conclusion we formulate a generalization of Theorem 3.2.4 due to P. Hubert ([Hub]) to 
rational polygons. Let the angles of a rational billiard k-gon be irrrii/r where mi, m^, r are 
coprime. Encode a billiard trajectory by the sides it reflect in. 

Theorem 4. If a trajectory is dense on the corresponding invariant surface of the billiard flow 
then the complexity of its encoding sequence 

p(n) = (k — 2)rn + 2r 

for all sufficiently large n. 

In the case of a square one gets p(n) = 4n + 4 which appears to be different from the result of 
Theorem 3.2.4. The reason for this discrepancy is that the encoding in that theorem was different 
from the present one: parallel sides of a square corresponded to the same symbol in the cutting 
sequence. 

3.5 Rational Billiards and Quadratic Differentials 

A recent breakthrough in the study of rational billiard polygons is due to applications of 
Teichmuller theory. A wealth of results obtained by S. Kerckhoff, H. Masur, J. Smillie and W. 
Veech relies heavily on the rather technical theory of quadratic differentials (see, e.g. [Str]). We 
are not in a position to discuss these results in detail; we will only outline some of them. Let us 
remark that this theory is specific for the two dimensional case - no analogs are known so far for 
higher-dimensional billiards. 

We saw in the previous section that the billiard flow in a rational polygon has invariant surfaces 
R. The surface R comes equipped with a special geometric structure: an atlas of charts mapping 
open subsets of R to R 2 with the change-of-coordinate functions of the form v — > v + c. The surface 
R has a finite number of singular points at which the charts are n-fold branched coverings of R 2 . 
This structure on R is a particular case of a quadratic differential, that is defined in a similar way 
with the only difference that the transition functions are v — > ±v + c and that at the singular 
points one has branched coverings of R 2 /(±l) (analytically, a quadratic differential on a Riemann 
surface is f(z)dz 2 with the singularities z k dz 2 ). 

A quadratic differential determines several other structures. Identifying R 2 with C one obtains 
a complex structure on R off singular points. This complex structure extends to all of R. A 
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quadratic differential defines a metric on R, flat off singularities; at singular points this metric has 
a cone type singularity with the cone angle a multiple of ir. The metric determines the area of 
R. A quadratic differential also defines a pair of foliations. The horizontal foliation is induced by 
the foliation of R 2 by horizontal lines, the vertical foliation is induced by the foliation of R 2 by 
vertical lines. These foliations have transverse measures determined by the quadratic differential. 
The measures are invariant: given two transverse intervals to the same leaf, the intersections with 
the leaves of the foliation define a mapping from one interval to another; this mapping is measure- 
preserving. A geodesic line on R is given by a line segment in each nonsingular coordinate chart; 
such a segment has vertical and horizontal components. Two isotopic geodesic lines have the same 
vertical and horizontal components. 

Given an element g 6 PSL(2,R) and an atlas {(pi} the products {g4>i} form a new atlas. 
Thus PSL(2,R) acts on quadratic differentials. One distinguishes the following one parameter 
subgroups: 

/ e*/ 2 \ ( cos 6 sin 6 \ , f 1 s\ 

9t= {0 e"*/ 2 J' re= {- S m9 costfj' hs = [o lj' 

referred to as the geodesic, circular and horocyclic flow, respectively. Notice that 

lim g t r sexp (_t)5_ t = h s 

(a direct computation with matrices). 

In Teichmuller theory one considers the Teichmuller space of complex structures on a sur- 
face modulo its diffeomorphisms isotopic to the identity. Similarly one considers the space Q of 
quadratic differentials with the area one modulo diffeomorphisms isotopic to the identity (this 
space is naturally identified with the unit cotangent bundle of the Teichmuller space). The map- 
ping class group M of diffeomorphisms modulo diffeomorphisms isotopic to the identity acts on 
this space. The quotient space QD = Q/M is the main object of study; the group PSL(2, R) acts 
continuously on QD. 

Many notions of ergodic theory extend to foliations with transverse invariant measure. Such 
a foliation is called ergodic if, given a saturated set (a union of the leaves), its intersection with 
any transverse disc has either full or zero measure. A foliation is called uniquely ergodic if there is 
a unique invariant transverse measure (which automatically is ergodic). 

Now we can formulate the results from [K-M-S] (see also [Arn]). 

Theorem 1. Given a quadratic differential q on a closed surface of genus not less than 2, for 
almost all 6 (in the sense of Lebesgue measure) the vertical foliation of rg(q) is uniquely ergodic. 

H. Masur ([M 2]) proved also that the Hausdorff dimension of the set of angles 9 for which 
the vertical foliation of rg(q) is not ergodic does not exceed 1/2. 

We already know that the billiard flows in different directions on the surface R are obtained 
one from another by the rotation r$. Applying the previous theorem to rational billiards one gets 
the following result. 
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Corollary 2. The billiard flow on the invariant surface is uniquely ergodic for almost all directions. 

An approximation of arbitrary polygons by rational ones, similar to the one used in Theorem 
3.4.2, leads to the next result. 

Theorem 3. There is a dense G$ subset in the space of polygons consisting of polygons for which 
the billiard flow is ergodic. 

We now very briefly outline the proof of Theorem 1 . Given a quadratic differential q consider 
its geodesic orbit gt(q). If it eventually leaves all compact sets in QD as t increases to infinity, q 
is called divergent; otherwise it is called recurrent. One proves that the set of angles 9 for which 
rg(q) is divergent is of measure zero. Then one considers the set S of 6 for which rg(q) is recurrent 
and its vertical foliation is not uniquely ergodic. There exists a closed set B in QD that contains 
all accumulation points of the orbits gt{q) for q € S. Each quadratic differential in B possesses a 
closed leaf of its vertical foliation (may be through singular points). 

If a is a closed geodesic of a quadratic differential q with the horizontal and vertical components 
h and v then the horizontal component of a with respect to h s {q) is h + sv. Hence a is vertical for 
at most one value of s. If (3 is isotopic to a it has the same horizontal and vertical components. 
Since there are only countably many isotopic classes of geodesies the intersection of the horocyclic 
orbit h s (q) with B is at most countable. Thus h s (q), —1 < s < 1 spends most of the time 
away from B. On the other hand, there is a sequence ti — > oo such that gti{q) — > B. Since 
h s g u (q) is approximated by g ti r S e X p(-t z ){<l) (see above), g u r e (q) is close to B for the majority of 
9 € [— e~ li , e~ li ]. This shows that is not a point of density for the set S. By changing coordinates 
one shows that S has no points of density, and therefore has zero measure. A subtlety in the actual 
proof is that all convergences involved are to be uniform, and the considerations should deal with 
compact sets. 

We would like to make a comment on this proof. Note that a result, concerning a given 
quadratic differential, is obtained by way of studying the space of classes of quadratic differentials 
and certain flows on it. The situation somewaht resembles that in knot theory after the works by 
V. Vassiliev: to construct invariants of an individual knot one studies the space of all knots. 

Another collection of results concern periodic orbits. We already saw that any rational polygon 
has a periodic billiard trajectory. The following theorem is due to H. Masur ([M 1]). 

Theorem 4. Given a quadratic differential q on a closed surface of genus not less than 2, there 
exists a dense set of angles 9 such that the vertical foliation of rg{q) has a closed nonsingular leaf. 

Corollary 5. For any rational billiard polygon there is a dense set of directions each with a 
periodic orbit. 

The following result from [B-G-K-T] is a strengthening of the previous one. 

Theorem 6. Periodic points are dense in the phase space of the billiard flow in a rational polygon. 

We reiterate that it remains unknown whether an arbitrary polygon has a closed billiard 
trajectory. 



71 



The next result by Masur ([M 3]) concerns the growth function of periodic billiard trajectories 
in rational polygons. Let N(t) be the number of strips of periodic trajectories of length not greater 
than t. 

Theorem 7. For any rational polygon there exist constants c and C such that for sufficiently 
large t the inequalities hold: ct 2 < N(t) < Ct 2 . 

W. Veech proved that the growth function is asymptotically quadratic 

N(t ) „ t2 
{ ' AreaP 

for the following polygons P: regular polygons and isosceles triangles with equal angles ir/n ([V 
1,2]). Veech found explicit formulas for the coefficients C(P) in each case. A similar result: the 
growth function is asymptotically quadratic, was proved for almost integrable polygons by E. 
Gutkin ([Gu 2]). 

Another remarkable result by W. Veech ([V 1,2]) strengthens Theorem 3.4.1 for regular poly- 
gons. 

Theorem 8. Any billiard trajectory in a non-exceptional direction is infinite or semi-infinite and 
uniformly distributed in the regular billiard polygon. Any billiard trajectory in an exceptional 
direction, that does not hit a vertex, is periodic. 

3.6 Miscellanea 

First, we discuss a reduction of the dynamics of point-masses in the line with elastic reflection 
to that of billiards. Consider two points with masses mi and 777.2 in the half-line x > (so if a point 
hits the " wall" x = its velocity changes sign) . The configuration space of this system is given by 
the inequalities < X\ < x 2 , where x\ and x 2 are the coordinates of the two points. Rescale the 
variables: 1,2. The configuration space is now the angle arctan \fm\Jm~2. 

fig. 67 

Consider a collision of the two points. Let 7>i,t>2 be the velocities before, and u\,U2 - after 
the collision. Then the conservation of momentum and energy reads: 

777iUi + 777 2 U2 = 777iT;i + 777 2 772, 
777i7if 777 2 7i2 _ TOiff 7772 7;| 

2 + 2 ~ 2 + 2 
In the rescaled coordinates the velocities are rescaled by the same factor, so: 

-2 , -2 -2 , -2 
U l + U 2 = V l + V 2 - 
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The second equation says that the length of the velocity vector (v 1,1)2) does not change after the 
collision. The first one says that the dot product of the velocity vector with the vector (y/mi, ^Jm 2 ) 
does not change as well. This vector is orthogonal to the boundary line of the configuration space: 
Xi/y/ml = X2I \J m 2 - Hence the configuration trajectory reflects in this line according to the billiard 
law. Likewise one considers a reflection in the wall x = to conclude that the system is isomorphic 
to the billiard in the configuration angle. Unfolding the trajectory yields the following conclusion: 
the number of collisions is always finite, and does not exceed the number 

7T 

arctan ^/mi/m 2 

fig. 68 

Similar considerations reduce the dynamics of several particles in the line, half-line or a segment 
to that of the billiard in a higher dimensional polyhedral angle or a polyhedron. 

The billard in a convex polyhedral angle was considered by Ya. Sinai in [Si 2]. The result is 
similar to the one in the plane. 

Theorem 1. The number of reflections of any trajectory is finite; it is bounded by a constant 
that depends on the angle only. 

Proof. We follow [G-Z]. Consider the unit sphere, centered at the vertex of the polyhedral 
angle, and project the angle onto the sphere from its center. The projection is a convex spherical 
polyhedron P, and a billiard trajectory in the angle projects to a billiard trajectory in P (in the 
spherical metric, of course). Unfolding a trajectory in the angle one gets a line in space, whose 
projection on the sphere is half a great circle. Hence the total length of the billiard trajectory in 
P equals it. 

The proof is by induction on the dimension. Consider the union of the e-neighbourhoods of 
the faces of P whose codimension is 2 or greater. Inside each neighbourhood the billiard is reduced 
to the one in a lower dimensional angle. Therefore the number of reflections of the billiard ball in 
each neighbourhood, until the ball leaves it, is uniformly bounded. The union of the codimension 1 
faces with the e-neighbourhoods removed is disconnected, and consists of the union of pieces of the 
faces. The distance between two such pieces is bounded away from zero by a constant c, depending 
on the polyhedron and e. Hence the billiard ball will travel for the length of at least c between two 
different e-neighbourhoods, and since the length of the trajectory is ir, the number of such "trips" 
does not exceed tt/c Thus the total number of reflections is uniformly bounded. Q.E.D. 

fig. 69 

The figure illustrates the situation in the 2-dimensional sphere, corresponding to a polyhedral 
angle in 3-space. Another proof of this result can be found in [Sev]. 
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The next example, due to G. Galperin ([Ga]), provides a non-periodic billiard trajectory that 
is not dense in a convex polygonal billiard table. 

fig. 70 

Consider a centrally symmetric hexagon such that the perpendicular to the side AF bisects 
the angle BAC, and the perpendicular to BC bisects the angle ABF. Then a vertical ray becomes 
vertical again after two reflections. Projecting the vertical rays onto the segment XY, the second 
iteration of the billiard transformation, acting on the rays, induces the exchange of the intervals 
XZ and ZY in it (one needs the following inequalities to hold: cot 2a > tan/3, cot 2(3 > tana). 
An exchange of two intervals is the same as a rotation of a circle; for generic a and (3 this rotation 
is irrational. Its orbits are infinite and dense, so the vertical billiard trajectories are dense in the 
hexagon. Extend the non-vertical sides of the hexagon to form a parallelogram whose triangular 
"ears" are never visited by any vertical trajectory. 

A question of interest is whether every triangle has a periodic billiard trajectory. As far as an 
acute triangle is concerned, the following "Fagnano" trajectory is, probably, known to the reader 
from his/her high-school years: it is an inscribed triangle whose vertices are the feet of the triangle's 
altitudes. 

fig. 71 

It is unknown whether a periodic trajectory exists in every obtuse triangle. Following [G-S-V] , 
we mention several explicit constructions of periodic billiard trajectories. 

The first construction concerns the triangles whose acute angles a and f3 satisfy the relation 
ka = n(3 < ir/2 for some positive integers k and n. After {k — 1) reflections about one vertex and 
n reflections about another vertex two sides become parallel. Their common perpendicular, being 
folded back, is a periodic trajectory, perpendicular to the sides of the triangle. 

fig. 72 

Another kind of periodic trajectory is shown in the next figure. The acute angles of this 
triangle satisfy a + n(3 = tt/2, 2a + (3 > tt/2. 

fig. 73 

The above trajectories are not stable in the sense discussed in Section 3 (i.e., they are destroyed 
by a small perturbation of the triangle). The last construction is free from this shortcoming. Let 
the acute angles satisfy the inequalities 

K — (3 , 7T . , . TT — a n TV . . „ 

—J± <ka <-<(k + l)a, — ^— <n(3<-<(n + 1)0. 
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Make (k — 1) reflections about one vertex and (n — 1) about another one. One obtains an acute 
triangle, whose Fagnano billiard trajectory folds back to a periodic trajectory in the original tri- 
angle. 

fig. 74 

Finally, consider right triangles. If it is a rational triangle, we know from Section 3.4 that 
almost every trajectory, perpendicular to a side, returns to this side in the same direction, and 
therefore is periodic. Remarkably, the same happens in the irrational case - see [C-H-K]. 

Theorem 2. Given a right triangle with ir-irrational acute angles, almost every (in the sense of 
measure ) billiard trajectory, that starts at a side of the right angle in the perpendicular direction, 
returns to this side in the same direction. 

fig. 75 

Proof. Let a be an acute angle of the triangle. Reflect the triangle in the sides of the right 
angle to obtain a rhombus R. The study of the billiard in the triangle reduces to that in the 
rhombus. 

fig. 76 

Construct the two dimensional invariant subspace of the phase space of the billiard flow in 
the rhombus, corresponding to the beam of horizontal trajectories which start at the upper half 
of the vertical diagonal. This is done as described in Section 3.4: consider the disjoint union of 
rhombi obtained from R by the action of the group 0(R), and identify their sides pairwise in an 
appropriate manner. One obtains a noncompact surface (partially) foliated by trajectories from 
the beam. This foliation has an invariant transversal measure ("width of a beam"). 

Each rhombus involved is obtained from R by a clockwise rotation through the angle 2na, n 6 
Z; such a rhombus will be refered to as the n-th rhombus R n (thus R = Ro). A trajectory from the 
beam may leave the n-th rhombus through one of its two sides; call a side positive if the trajectory 
enters the (n + l)-st, and negative if it enters the (n — l)-st rhombus. 

fig. 77 

One wants to show that almost all trajectories return to Ro (where they stop at the vertical 
diagonal). Since a is 7r-irrational, for every e > there exists n > such that the vertical projection 
of the positive side of R n is smaller than e - see Section 3.1 on irrational circle rotations. Hence 
the set of trajectories that make it to R n +i has measure less than e. 
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The rest of trajectories is bound to stay in R ,...,R n ; call the set of these trajectories S. The 
union of the rhombi through n is compact, and the Poincare recurrence argument applies as in 
Section 3.4. Namely, almost every trajectory in S is recurrent; since it makes only finitely many 
different angles with the vertical diagonal of the rhombus R, it will become perpendicular to it, 
that is return to Rq. 

Said differently, the total width of the beams that constitute S is positive, and the total area 
of the rhombi through n is finite. Since beams cannot overlap, almost all trajectories from S 
come back to R . 

Since the above e is arbitrary small, the result follows. Q.E.D. 

It seems that this argument can be extended with appropriate changes to other polygons 
(perhaps all?!) 

Another result on periodic trajectories in right triangles is due to G. Galperin and A. Zvonkin 
([Ga-Zv]). 

Theorem 3. There exists a periodic billiard trajectory through every point of a right triangle. 
Moreover these trajectories can be chosen so that their lengths are bounded by a constant depending 
on the triangle only. 

Concerning the numerical study of the billiard in a triangle we also refer to [Ru] , [C-H-K] and 
[Bos]. 
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4. Dual Billiards 



The topic of this chapter is the lesser known dual billiard transformation. We define it in 
Section 1 and discuss its basic features such as the area-preserving property. We also comment 
on the relation between billiards and dual billiards via projective duality. Section 2 gives an 
approximation of the dual billiard dynamics far away from a dual billiard; if the billiard is smooth 
enough and strictly convex the map has invariant curves, and all orbits are bounded and separated 
from the table. We introduce the area spectrum of a dual billiard and obtain some results similar 
to those from Section 2.7. Section 3 contains several dynamical proofs of the classical Poncelet's 
theorem from projective geometry; the first one is by way of dual billiards in the hyperbolic plane. 
Section 4 concerns polygonal dual billiards; it provides a sufficient condition for all orbits to be 
bounded. We also describe the dual billiard dynamics for an affine-regular pentagon. The last 
section contains the definition of higher dimensional dual billiards; we prove that this map has 
periodic trajectories of all odd prime periods. 
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4.1 Definition, Area-Preserving Property and Generating Function 

Dual billiards (also known as outer billiards) are in many ways similar to billiards. The game 
of dual billiard is weird in a sense: the "balls" move outside of the dual billiard "table", and the 
motion is defined for discrete time only. Let 7 be an oriented strictly convex closed curve in the 
plane (a dual billiard table). Given a point x outside of 7 draw the segment xy, y 6 7 of the 
supporting line to 7, whose orientation from x to y agrees with that of 7, and extend it through y 
to the point T(x) such that dist (T(x),y) = dist (y, x). The map T of the exterior of 7 to itself is 
the dual billiard transformation. 

fig- 78 

If 7 is convex but has straight segments the dual billiard transformation and its inverse are 
defined off the extensions of these straight segments. This is analogous to billiards: one does not 
define an extension of a trajectory through a corner of a billiard curve. If 7 is not convex T is 
defined as a multiple- valued map. 

We start with a fundamental property of dual billiard maps. 

Lemma 1. T is area-preserving. 
fig. 79 

Proof. Let 8 be a convex curve; fix c > and consider the one-parameter family of straight 
lines that cut off segments of area c from 8. Let 7 be the envelope of this family. Claim: the 
segment of any line from this family determined by its intersection with 8 is bisected by the point 
of tangency with 7. Indeed, suppose OA > OB in figure 79. Let A'B' be a nearby line from 
the family and let O' = AB n A'B' . Then O'A > O'B. The central symmetry in O' sends the 
"triangle" BO'B' inside AO' A'. On the other hand, Area(BO'B') = Area(AO'A'), which is a 
contradiction (this elementary argument is contained in [F-T]). 

Consider now two nearby curves 8 and Si— see figure 79. It follows from the previous argument 
that the shaded areas are equal. Hence T is area preserving. Q.E.D. 

Corollary 2 (of the proof). Given a convex curve 8 there exists a one-parameter family of 
curves 7 such that 8 is an invariant curve for the dual billiard transformation with respect to 7. 
The curves 7 are the envelopes of the segments of equal areas. 

The reader will recognize in this construction the dual billiard analog of the string construction 
from Section 2.1. Notice that the "duality" interchanges length and area. We will comment on this 
later. Another consequence of the proof concerns periodic trajectories of the dual billiard map. 
Connecting consecutive points of such an orbit one obtains a circumscribed ra-gon whose sides are 
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bisected by their tangency points to 7. Such a polygon may make r turns about 7, 1 < r < re/2; 
the number of turns is called its rotation. 

fig. 80 

It follows from the proof of Lemma 1 that re-periodic orbits correspond to extrema of areas of 
ra-gons circumscribed about 7. For a smooth strictly convex curve this area functional attains its 
minimum on the set of circumscribed n-gons with the rotation r. 

Corollary 3. If a dual billiard curve is smooth and strictly convex then for any coprime numbers 
1 < r < n/2 there exists an n-periodic orbit of the dual billiard map with the rotation r. 

This corollary also follows from the general results concerning area-preserving twist maps 
(Sections 2.6 and 2.10). Introduce the coordinates x(a, r) in the exterior of 7: a is the angle made 
by the positive tangent line to 7 in the direction of x with a fixed direction in the plane, and r is 
the distance along this line from 7 to x. 

fig. 81 

First, we observe that T is a twist map with respect to the "vertical" foliation a = const - see 
figure 81. Secondly, let T(a,r) = (/3,R), and denote the area of the shaded curvilinear triangle in 
figure 81 by S(a,/3). Let uj be the standard area form in the plane. The next result follows from 
a direct computation. 

Lemma 4. 

lo = rdr A da 

and 

r 2 _ dS{a,f3) R 2 _ dS{a,(3) 
~2 ~ da ' T ~ dp ' 

It follows that S(a,(3) is a generating function of the map T: if A = r 2 da/2 then d\ = uj 
and T*A — A = dS. We again see that T is area preserving, and that the area of the circumscribed 
polygon corresponding to a periodic orbit is a particular case of the symplectic invariant, introduced 
at the beginning of Section 2.7. We also notice for the second time that area plays the role in the 
dual billiard problem analogous to that of length in the billiard one. 

To compare billiards and dual billiards consider both on the sphere. We use the projective 
duality, interchanging points and lines (that is, great circles): to a pole its oriented equator corre- 
sponds. A point on a curve 7 is sent to a tangent line to the dual curve 7*, which, by definition, 
consists of the tangent lines to 7. Notice that the angle between two lines equals the distance 
between the dual points. 

Lemma 5. Billiards and dual billiards are projective dual in the sphere. 
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fig- 82 



Proof. The billiard map acts on oriented lines: to a ray the reflected ray corresponds. 
Consider a moment of reflection in 7. The dual configuration consists of a tangent line to 7* and 
two points on it. Since Angle(a,b) = Dist(A, B) the billiard rule of angles translates to the dual 
billiard rule: dist (A,L) = dist (L, B). Q.E.D. 

Closed billiard trajectories are extrema of the length functional on inscribed polygons. There- 
fore closed dual billiard trajectories are extrema of the "sum of angles" functional on circumscribed 
polygons. By the Gauss-Bonnet theorem this sum equals, up to constants, the area of the polygon. 
Whence the length - area duality. However in the flat case (which is the limit of the spherical 
one: the radius goes to infinity) the two problems become different: dual billiards are equivariant 
with respect to the group of afnne transformations of the plane, while billiards are equivariant 
with respect to the smaller group of similarities. Another important difference is that there is no 
configuration space for the dual billiard transformation. 

We mention the dual billiard analog of the mirror equation from Section 2.4 found by E. 
Gutkin and A. Katok ([Gu-K 1]). Let 5 be an invariant curve of the dual billiard transformation 
T with respect to a curve 7, and let p(y) be the curvature radius of 7 at point y 

fig. 83 

Lemma 6. 

cot a + cot /3 = J^LL. 

r 

This lemma is proved by a direct computation which we omit. 

The results and constructions of this section had been discovered and rediscovered by several 
authors. Corollary 3 is found in [Da]; the construction of dual billiards and their area-preserving 
property were considered by J. Moser in the framework of KAM theory ([Mo 4]). Ph. Boyland 
lectured on dual billiards in the late 80-s; his paper on the subject is in preparation. We also refer 
to [Ta 1,2] for a general discussion of dual billiards. 

4.2 Invariant Curves, Integrability, Area Spectrum 

A natural question to ask about dual billiards is whether orbits can escape to infinity or "fall" 
on the dual billiard curve. If the curve is sufficiently smooth (C 7 is enough) and has nonvanishing 
curvature the negative answer is provided by KAM theory. In a vicinity of 7 the dual billiard map 
T is a small perturbation of the shear map (a, r) — > (a + 2r, r) , therefore T possesses an abundance 
of invariant curves near the dual billiard curve. These curves separate orbits from 7. 

Consider the situation near infinity. We use the variables a and p = 1 jr as the local coordi- 
nates. In these coordinates the second iteration T 2 is again a small perturbation of a shear map 
near the curve p = 0. Hence there exist invariant curves outside of any disc containing 7, and 
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these curves prevent orbits from escaping to infinity. J. Moser ([Mo 6]) considered the dual billiard 
transformation as a crude model for planetary motion; thus for a sufficiently smooth dual billiard 
curve this "planetary motion" is stable. 

One can be more specific concerning the dual billiard dynamics near infinity. Let 7 be a 
strictly convex smooth closed oriented curve. Denote by y{a) the point of 7 at which the tangent 
direction to 7 makes the angle a with a fixed direction in the plane. Set v{a) = y{a + ir) — y{a). 
We extend the definition of v{a) to strictly convex piecewise smooth curves, replacing tangent 
lines by the supporting ones. v{a) is a continuous odd vector function of a. We further extend 
the definition to convex piecewise smooth curves, in particular, to polygons. Let I be a straight 
segment of 7 in the direction a. Then v(a — e) and v(a + e) are defined for small e, and v(a) has 
a discontinuity of the first kind at a. 

fig. 84 

Consider in another copy of the plane with polar coordinates (a, r) a homogeneous vector field 
V(a,r) = v(a). The following result is contained in [M-T]. 

Lemma 1. The integral curves of the field V are closed homothetic curves, centrally symmetric 
with respect to the origin. If 7* is one of these curves then corners of 7* correspond to straight 
segments of 7, and straight segments of 7* - to corners of 7. Up to a dilation, 7*** = 7*, and if 7 
is centrally- symmetric then 7** = 7. The flow defined by the vector field V satisfies the Kepler 
law: area, swept by the position vector of a point, is linear in time. 

Proof. Choose an origin inside 7 and let p(a) be the support function, that is the distance 
from the origin to the supporting line of 7 in the direction a. The point of tangency of this 
supporting line with 7 has the coordinates: 

p(a) sin a + p'(a) cos a, —p(a) cos a + p'(a) sin a 

(see, e.g. [Sa] or make a direct computation). Let d(a) = p(a + tt) + p(a) be the width of 7 in the 
direction a. It follows that the vector v(a) has the components 

—(d(a) sin a + d'(a) cos a), d(a) cos a — d'(a) sin a. 

fig. 85 

Let a trajectory 7* of the vector field V be given by a function r(a) in polar coordinates. 
Since x = r cos a, y = r sin a one concludes that 

r'(a) sin a + r(a) cos a d'(a) sin a — d(a) cos a 
r'(a) cos a — r(a) sin a d'(a) cos a + d(a) sin a 
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Hence r(a) = const /d(a) is the polar equation of 7*. 

Geometrically it means that 7* is obtained from 7 in two steps: symmetrize, i.e., replace p(a) 
by p(a) + p(a + it); polar dualize, i.e., replace p(a) by l/p(a). In invariant terms, polar duality 
assigns to a point other than the origin the affine line in the dual space that consists of the linear 
functionals whose value at this point equals 1. From this description all but the last statement 
follow. 

To prove the Kepler law we compute the cross-product of the velocity vector with the position 
vector and use the equation r(a) = const/ d(a): 

d'(a) cos a + d{a) sina, const cosa/d(a) _ CQns ^. 
d'(a) sin a — d(a) cos a, const sin a/<i(a) 

Q.E.D. 

Note that the position of the origin in the previous consideration is irrelevant. 
The following figure shows several curves 7 and the corresponding curves 7*, 7**. If 7 is a 
semi-circle, 7* consists of two arcs of equal parabolas intersecting at right angle. 

fig. 86 

The significance of the above described continuous motion is that is provides an approximation 
of the second iteration of the dual billiard map T near infinity. Choose an origin O inside 7 and 
let x be a point outside of 7 at distance d from the origin. Consider the orbit of x under T 2 , and 

let X Q X . X ]^ • • • • 5 X f\, 

be its segment that makes one turn about the origin: 
Angle(x O xi) + ... + Anglc{x n _\ O x n ) < 2ir, 

Angle(x O xi) + ... + Angle{x n _i O x n ) + Angle(x n O x n+i ) > 2ir. 

Let T be the dilation with the center O and the coefficient 1/d. Consider the motion along the 
vector field V, and let {y t } be the closed trajectory of the point T(x). Normalize time so that 

yi = yo- 

Theorem 2. For all 1 < i < n, 

Dist (ui/ n , T(xj)) ^0 as d — > 00. 

The proof is a straightforward estimation, and we omit it (see [M-T]). The following figure 
shows that for large d the map T 2 is almost a translation through the vector 2v(a). 

fig. 87 

If the smoothness of a dual billiard curve is less than required by KAM theory one cannot 
guarantee the existence of invariant curves " at infinity" . In such a case almost nothing is known 
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concerning whether orbits may escape to infinity (however, a sufficient condition for stability in 
the case when the dual billiard curve is a polygon is known; we will discuss it in the next section). 
There is a strong computer evidence and an heuristic argument to the effect that if the dual billiard 
curve is a semi-circle then some orbits go to infinity, and the set of such orbits has a nonvoid interior 
(see [M-T]). The dual billiard transformation is discontinuous in this case. 

For the dual billiards, like for the inner ones, the shape of the table imposes restrictions on 
the position of invariant curves of the corresponding dual billiard map. 

Theorem 3. If a dual billiard curve 7 has a point of infinite curvature (say, a corner) then there 
are no invariant curves sufficiently close to 7. 

This dual billiard analog of Mather's Theorem 2.9.4 was stated in Ph. Boyland's MSRI talk 
in 1989; see [Gu-K 1] for a proof. The next figure illustrates the idea of Boyland's proof. 

fig. 88 

Let x be sufficiently close to a point of infinite curvature. Note that T is a reflection in a point 
in the linear approximation. Let u be a tangent vector at x in the direction of the tangent line 
to the dual billiard curve, and v be its image under the derivative of the map T 2 . In the (a,r) 
coordinates, u is vertical and v is deviated from the vertical in the negative sense. 

Suppose there is an invariant curve C through x; according to Birkhoff's theorem (Section 2.9) 
it is a graph: C = {(a,r(a))}. Since T deviates to the right, consecutive images of of the vertical 
vector u under the derivative of T are contained in the cones, spanned by an upward vertical vector 
and a tangent vector to C with a positive first component. In particular, v cannot have a negative 
first component. 

Alternatively the result can be deduced from the "dual billiard mirror equation" of Lemma 
4.1.6 (see [Gu-K 1]). The work of Gutkin and Katok contains explicit estimates for the size of the 
domain free from invariant curves in the spirit of Theorem 2.9.6. 

Ph. Boyland also constructed a differentiable dual billiard curve 7 with a crash orbit of the dual 
billiard map, i.e. an orbit that converges to a point on 7. However if 7 is C 1 and the derivative of 
its curvature is bounded such an example is impossible (compare Halpern's construction discussed 
at the end of Section 1.7). 

We now turn to the question of integrability. Clearly the dual billiard transformation with 
respect to an ellipse is integrable: its invariant curves are concentric homothetic ellipses. The dual 
billiard counterpart to Birkhoff's conjecture (Section 2.4) says that this is the only integrable case. 
This conjecture is not proved; a partial result in this direction is the following theorem from [Ta 
3]. 

Theorem 4. Let two smooth strictly convex curves be given such that the corresponding dual 
billiard transformations commute in their common domain (that is sufficiently far away from both 
curves). Then the curves are concentric homothetic ellipses. 
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In conclusion of this section we consider the areas of circumscribed polygons corresponding 
to periodic orbits of the dual billiard map. Call the set of these areas the area spectrum of a 
dual billiard curve. An interesting problem is to what extent the area spectrum determines the 
curve, in particular, is it true that it uniquely determines the curve up to an area-preserving affine 
transformation. Another question to ask is whether the area spectrum is related to the spectrum 
of some differential operator in a way similar to the relation between the length spectrum and the 
spectrum of the Laplace operator (Section 2.7). Notice that such an operator, if it exists, must be 
invariant under area-preserving affine transformations. 

fig. 89 

To formulate the next result, which is the dual billiard analog of Theorem 2.7.1 and which is 
proved in a similar way, we make a short digression to discuss affine length (see, e.g. [Sp]). Let 7 
be a smooth strictly convex plane curve. Fix a point x 6 7 and let u be a tangent vector to 7 at 
x. Consider a tangent vector field to 7 that takes the value u at x, and let x e be the point of 7 to 
which the flow of the field for time e takes x. The area A(u, e) of the segment, bounded by 7 and 
the chord xx e , is of the third order in e. Hence 

B x {u) = J™— — 

is a cubic form on the tangent bundle to 7, and B 1 / 3 is a 1-form. The integral of this 1-form 
over the curve is called its affine length. By the very definition the affine length is invariant under 
area-preserving affine transformations of the plane. Also, by definition, the affine length has the 
degree of homogenuity 2/3, that is it is multiplied by k 2 / 3 under a dilation with the coefficient k 
(in down-to-earth terms, one measures the affine length in meters 2 ^ 3 ). An equivalent definition, 
more convenient for computations: parametrize 7 by a parameter t, a < t < b, so that [7', 7"] = 1 
([ , ] being the cross-product). Then the affine length of 7 is b — a, and the function [7", 7"'] is 
called the affine curvature. 

It is interesting that the isoperimetric inequality for convex curves in affine geometry goes in 
the "wrong direction": 

(Affine Length) 3 < M 2 Area, 

with the equality for ellipses only (see [B-Z]). 

We now formulate a result from [Ta 1]. Let A n denote the area of a simple polygon, circum- 
scribed about a smooth strictly convex dual billiard curve 7, corresponding to an re-periodic orbit 
of the dual billiard map. 

Theorem 5. The asymptotic expansion holds: 

A n ~ a + -J + —7 + ••• + -jj + 

n z n 4 n zi 
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where ao is the area bounded by 7, and 

ai = — (Affine Length 0/7) 3 . 

In view of the affine isoperimetric inequality, 3ai < 7r 2 a , one can thus recognize an ellipse by 
its area spectrum. 

4.3 Poncelet's Theorem 

Poncelet's theorem is one of the most beautiful results of classical projective geometry. Since 
it is close to both billiards and dual billiards we choose to discuss it here. Given two nested ellipses 
70, 71 in the plane one plays the game illustrated in the figure: choose a point x G 71, draw a 
tangent line to 70 through it, find the intersection y with 71, and iterate, taking y as a new starting 
point. The statement is that if x returns back after a number of iterations, then every point of 71 
will return back after the same number of iterations. 

fig. 90 

We will give a few proofs of this theorem; the reader, interested in its history and the classical 
proofs, is referred to [B-K-O-R] . 

Let us start with the dual billiard approach ([Ta 4]). Consider an ellipse in the plane, and 
take its interior as the Klein-Beltrami model of the hyperbolic plane. The distance between points 
is given by dist (x,y) = | log[x, y, b, a] |, where [ ] denotes the cross-ratio, and straight lines are 
represented by chords of the ellipse. The hyperbolic metric determines an area form with infinite 
total area. 

fig. 91 

Let 7 be a strictly convex smooth closed curve in the hyperbolic plane. One defines the 
dual billiard transformation T of its exterior in the same way as in Section 4.1 (using hyperbolic 
distances, of course). The arguments of Section 4.1 still apply, so T is area preserving. 

Given two nested ellipses 70 and 71 consider the one-parameter family of conies 7 t , called a 
pencil, which pass through the four intersection points 70 PI -71 (imaginary in our case). The curves 
7i foliate the annulus between 70 and 71. Choose an ellipse from the pencil, which contains 71, 
and call it 700. Let T be the dual billiard map with respect to 70 in the hyperbolic plane, modelled 
in the interior of 700 . 

Lemma 1. T is integrable: its invariant curves are ellipses from the pencil. 

Proof. Let / be a line in the plane; its intersections with conies from a pencil 7 t define an 
involution on I. We claim that this involution is a projective transformation of the line (Desargues' 
theorem - see [Be 1]). 
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fig. 92 



Indeed, applying a projective transformation of the plane, we make the conies 74 concentric. 
Take the center as the origin. Then the pencil consists of conies 

7t = {x I < A t x, x >= 1}, 

with A t = A + tE where A and E are selfadjoint operators. Let I be tangent to 70 at point x and 
u be a tangent vector to 70 at x. Parametrize I by a parameter s so that points of I are x + su. 
The intersection I n 7t is given by 

< (A + (a; + su), 2 + su >= 1. 

Since < Ax, x >= 1 and < Ax, u >= the previous equation is rewritten as 

s 2 < (A + tE)u, u > +2st < Ex, u> +t < Ex, x >= 0. 

It follows that 

11 < Ex, u > 

— + — = -2 — E r J 

Si s 2 < Ex, x > 

independently of t; here S\ and s 2 are the roots. Thus the correspondence Si <-> s 2 is fractional- 
linear, that is projective. 

To finish the proof consider the next figure. The above involution sends x to y and a to b, 
preserving c. Since it preserves the cross-ratio, dist (x,c) = dist (c,y). Q.E.D. 

fig. 93 

Back to Poncelet's theorem. Since T is integrable, its restriction to each invariant curve, 71 in 
particular, is a translation in an appropriate afline coordinate therein (Section 1.8). Thus if some 
orbit is n-periodic, so is any other orbit. 

Two other results readily follow. First, consider a number of ellipses from one pencil: 7, 7', 7", 
etc, and let 7 contain all the others. Pick a point x 6 7, draw a tangent line to 7', find its 
intersection with 7, draw a tangent line to 7", etc. Then if x returns back after a number of 
iterations, any initial point of 7 does (this generalization was known to Poncelet). This is again a 
consequence of the fact that T,T' ,T' , etc, are translations in the afline parameter on 7. 

Secondly, consider three nested ellipses 7, 7', 7" from a pencil, and identify the interior of the 
outer one 7 with the hyperbolic plane. Then the dual billiard transformations T' and T" commute. 
This follows from Corollary 1.8.3. 

fig. 94 

Another elegant proof of Poncelet's theorem is contained in [Ki]. One constructs a smooth 
invariant measure on the outer ellipse T, invariant under Poncelet's map x — > y (see figure 95). 
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Applying an affine transformation we may assume that V is a circle. Let the desired measure be 
f(x) dx, where dx is the Lebesgue measure on T. 

fig. 95 

Denote by i? 7 (x) and L 7 (x) the length of the right and left tangent segments from x to 7. 
Consider a point x±, e-close to x. Since the segment xy makes equal angles with F, the proportion 
holds: 

Arc X\X Ry(x) 

hm — = — V^r- 

e^o Arc y x y L 7 (y) 

The measure of the x-arc is its length times f(x), and likewise for y. So the measure is invariant 
if and only if 

f(x) L^y)' 

If 7 happens to be a circle the right and left tangent segments are equal: R 7 (x) = L 7 (x). Denote 
this common value by D 1 {x). If 7 is not a circle, let A be an affine transformation that takes 7 to 
one. We have 

Rj(x) Raj(Ax) DA-y{Ax) 
-Ml/) ~ L Al {Ay) ~ D Al (Ay) ' 

Thus f{x) = l/DA-y{Ax) is the desired function. 

Finally, choose a coordinate t in which f(x) dx = dt. Then Poncelet's map is a translation 
t — > t + c, and Poncelet's theorem follows. 

A variation of this argument is found in [Ko 1]. If both ellipses are circles, one proceeds as 
above. We claim that there exists a projective transformation of the plane that takes the given 
ellipses 7 and T to two circles. First, apply a projective transformation to make the ellipses 
concentric. Then apply an affine transformation to make V into a circle. Let the plane be the 
horizontal plane in 3-space, and consider the sphere whose equator is Y. Let 8 be its meridian 
whose plane contains the major axis of 7. For a point x € 5 consider the tangent plane to the 
sphere at the point, antipodal to x. Project 7 and T onto this plane from x. 

fig. 96 

This projection carries T to a circle (any circle on a sphere goes to a circle under a stereographic 
projection), and 7 - to an ellipse 7', one of whose axes lies in the plane of 8 and another is 
perpendicular to it. If x is the south pole, the major axis of 7' lies in the plane of 5, and if x is 
close to the equator this major axis is perpendicular to this plane. By continuity there exists a 
position of x such that 7' is a circle. 

Finally we outline the proof by Ph. Griffiths and J. Harris via algebraic geometry ([G-H 1,2]). 
First, complexify the situation: both ellipses 7 and T are considered as conies in the complex plane 
C 2 . Consider the incidence set E that consists of pairs: (a tangent line to 7, a point of V on this 
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line). E is an algebraic curve in 7* x V = (CP )* x CP (the asterisk denotes the dual space), 
nonsingular because the intersection 7nT is transversal. The projection of E to Y is two-fold except 
for the four points of intersection 7 n T. This makes it possible to compute the Euler characteristic 
of E, which proves to be equal to 0. Hence E is a torus. 

fig. 97 

Poncelet's correspondence is the product of two involutions on E: the one, which interchanges 
the two points of intersection of a tangent line with T, and the one, which interchanges the two 
tangent lines to 7 from a point. These involutions are induced by automorhisms of the universal 
covering E = C. Each one is of the form z — > — z + c, and their composition is a parallel translation. 
Thus Poncelet's correspondence is a translation of the torus E, and Poncelet's theorem follows. 

4.4 Polygonal Dual Billiards 

A comparison of Chapters 2 and 3 clearly shows that billiards in polygons differ drastically 
from the ones in smooth strictly convex domains; likewise polygonal dual billiards are worlds apart 
from smooth stricly convex ones. To start with, if a dual billiard curve 7 is a convex polygon, the 
dual billiard transformation T and its inverse are defined off the set U which consists of the union 
of straight lines containing the extensions of the sides of 7, as well as its forward and backward 
orbits under T (this difficulty is analogous to the one encountered in the case of billiards: one has 
to ignore the trajectories that hit a corner of a billiard table). Let V be the exterior of 7 with the 
set Vo = \J^ = _ 00 T n {U) deleted. Since Vq is a countable union of lines the set V has full measure. 

The set V consists of two pieces: Vf - the set of periodic points, and - the set of points 
with infinite orbits. Let x € Vf, that is T n {x) = x for some n. The map T n is locally a parallel 
translation (if n is even) or a reflection in a point (if n is odd). It follows that a point, sufficiently 
close to x, is either n-periodic (for an even n) or 2n-periodic (for an odd n). The set Vf is open 
and its connected components are open polygons bounded by segments from Vq. The set may 
be empty: for example, it happens when the vertices of 7 belong to a lattice (see below). It is still 
unknown whether Vf may be empty for any polygon. 

In analogy with billiards in polygons (Section 3.3) call a periodic orbit of the dual billiard 
transformation stable if an arbitrary small perturbation of the dual billiard polygon leads to a 
deformation of this orbit but not to its destruction. Enumerate the vertices of a polygon counter- 
clockwise. Let x be a 2n-periodic point, and let S{x) = ...,i2n) be the sequence of vertices in 
which x makes successive reflections. The next statement is parallel to Lemma 3.3.1. 

Lemma 1. The orbit of x is stable if and only if the symbols in the list (ii,...,«2n) can be 
partitioned in pairs of equal symbols, so that each symbol from every pair once appears at an even 
position and once at an odd one. 

Proof. Choose an origin and let V{ be the position vector of the i-th vertex. Then T 2n x is 
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its parallel translation through the vector 

2(-V n + V l2 - V l3 + ... - + V i2n ) = 0. 

This relation persists under an arbitrary deformation of the vertices if and only if the terms cancel 
pairwise. Q.E.D. 

As in Section 3.3 it follows that for a generic dual billiard polygon all periodic trajectories are 
stable. 

The procedure of unfolding a billiard trajectory has a dual billiard analog too: choose a point 
x outside of 7 as a point of reference, draw a supporting line to 7 and reflect 7 at the vertex that 
lies on this line (we assume that the line does not contain a side). Iterating this reflection one 
obtains a "necklace" of polygons, congruent to 7, around x. This necklace is periodic if and only 
if the T-orbit of x is periodic, and it is bounded if and only if so is the T-orbit of x. The shape of 
the necklace is, roughly, that of the polygon 7* from Section 4.2. 

fig. 98 

J. Moser ([Mo 6]) asked for which polygons all orbits of the dual billiard transformation are 
bounded. A sufficient condition was found by A. Shaidenko and F. Vivaldi ([S-V]) and by R. 
Kolodziej ([Ko 2]); see also [Gu-S]. This condition is formulated in terms of the motion whose 
trajectories are the curves 7*, defined in Section 4.2. If 7 is a polygon the vector function v(a) is 
piecewise constant, each vector v(a) is a diagonal of 7, and 7* is also a polygon. 

fig. 99 

Let 7* have ./V sides (actually N is even because 7* is centrally symmetric), and let ti, t 2 , ...,ijv 
be the values of time it takes a point, moving along the vector field V from Section 4.2, to cross 
the sides of 7* (said otherwise, t{ is the ratio of the length of the i-th side of 7* and the length of 
the corresponding vector v). Since 7* is defined up to a dilation, these numbers are defined up to 
a common factor. Call a polygon quasirational if (t\ : t<i : ... : t^) € QP W_1 . The property of a 
polygon to be quasirational is affine invariant. 

Lemma 2. Any lattice polygon (i.e. a polygon whose vertices belong to a lattice in the plane) is 
quasirational. 

Proof. Without loss of generality assume that the vertices of 7 are rational points. Choose 
an origin O in the plane. The polygon 7* can be constructed as follows. Draw the lines through O 
parallel to the sides of 7. These lines have rational slopes, and they partition the plane into cones; 
the value of the vector function v(a) is constant in each cone. Choose a rational point A\ on one of 
the lines as the first vertex of 7*. To construct the next vertex A 2 draw the line through A\ in the 
direction of the corresponding vector v until its intersection with the next line through O. Since v 
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is a rational vector, A 2 is a rational point too. Therefore the vector A 1 A 2 is a rational multiple of 
v, which means that t\ 6 Q. Repeat the argument to conclude that t 2 6 Q, etc. Q.E.D. 

fig. 100 

Notice that a regular polygon is quasirational as well: by symmetry t\ = t 2 = ... = tjv- A 
regular n-gon is a lattice polygon only when n = 3,4,6. Thus the class of quasiregular polygons is 
bigger than that of lattice polygons. 

Theorem 3. All orbits of the dual billiard map, corresponding to a quasirational polygon, are 
bounded. 

There are two approaches to the proof of this theorem. Shaidenko and Vivaldi, as well as 
Kolodziej, show that outside every circle containing 7 there exists an invariant set of the dual 
billiard transformation that separates 7 from infinity. This set is a union of polygons, and it plays 
the role similar to that of invariant curves in the smooth case (see figure 101). Gutkin and Simanyi 
gave another proof based on the unfolding procedure. We outline their argument. 

fig. 101 

Proof. Start as in the proof of the previous lemma: choose an origin O and partition the 
plane into the cones C\, C 2 n by lines parallel to the sides of 7. Consider the set P of polygons, 
not containing O, obtained from 7 either by a parallel translation or by a reflection in a point. Let 
p G P be a polygon such that O does not belong to the extension of any of its sides. Define the 
necklace transformation W as the reflection of p in its vertex that lies on the left supporting line 
to p from O. Call the vertex of reflection the head of p and denote it by h(p). 

fig. 102 

One wants to show that if 7 is quasirational then all orbits of W are bounded. The polygon 
p G Pis uniquely characterized by the following data: the position of its head and the value of the 
function e(p) = ±1, according to whether p is a parallel translation of 7 or its central symmetric 
image. To each cone Cj a vector Vi corresponds, the value of the locally constant function v(a) in 
this cone. Thus if p and W(p) belong to Cj then the vector from h(p) to h(W(p)) equals V{. The 
sides of the polygon 7* are parallel to the vectors Vi . 

Let Ri, R 2 n be the boundary rays of the cones. Consider a ray Ri, and let Sf be the union 
of the heads of polygons p 6 P which intersect Ri and for which e(p) = ±1, correspondingly. Then 
and S~ are semi-infinite strips, bounded by the rays Ri and Ri + Vi, and truncated by certain 
polygonal lines. Let Si be the disjoint union of Sf~ and S~ , and let : Si — > Sj+i be the mapping 
induced by W. 
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fig. 103 



Consider the collection of parallel rays Ri+i, Ri+i — — 2vi,Ri+i — 3uj,...; their in- 

tersection with Sf 1 partition the strips into parallelograms. Let cij be the vector of the side of 
such parallelogram along the ray Ri, and let bi = + Vi be the vector of the other side of a 
parallelogram. It follows that the map F has the periodicity property: Fi(x + 2a^) = Fi(x) + 26j. 

fig. 104 

Suppose that 7 is quasirational. Without loss of generality assume that all numbers ti are 
integers. Then the triangle whose sides are ai and bi is similar to the triangle OAiA i+1 , where A, 
and A i+ i are the vertices of 7*. Thus OA i+1 = tfii = t i+ iai + i, and 

F,{x + 2Ua t ) = Fi{x) + 2Ubi = F^x) + 2t i+1 a i+1 . 

fig. 105 

It follows that the first return map : Si — > Si, which is a composition of the maps Fj, is 
also periodic: 

$(x + 2tiai) = $(x) + 2£i<n. 

Let LT 1 * 1 be the union of the first 2ti parallelograms, counting from the origin, entirely contained in 
S^ 1 . Then S^ 1 is an infinite union of parallelograms, congruent to n^, and an additional polygon, 
neighbouring O. Let II be the disjoint union of n + and II - . The map $ induces a transformation 
T of II such that for x G II 

= T(ar) + t(x)2tiai, 

with an integer i(x) > —1. Replace $ by to conclude that i(x) < 1. The same argument 
applies to any iteration of <&: 

$ n (x) =T n (x) + i n (x)2t iai ; i n (x) G {-1,0,1}. 

Thus the $-orbit of a point from II stays a bounded distance from II; by periodicity all orbits of 
$ are bounded. Q.E.D. 

Corollary 4. If the vertices of a dual billiard polygon 7 belong to a lattice then each orbit of the 
dual billiard map is periodic. 

Proof. We know from Lemma 2 that 7 is quasirational. By Theorem 3 all orbits are bounded. 
The group of motions of the plane generated by the reflections in vertices of 7 is discrete. Each 
orbit of the dual billiard map is contained in an orbit of this group. Being discrete and bounded 
it is finite. Q.E.D. 
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Consider a polygon such that all orbits of the dual billiard map are bounded. Let x £ be a 
point with an infinite orbit, and let S(x) be its encoding sequence of vertices in which it undergoes 
consecutive reflections. 

Lemma 5. S(x) is an aperiodic sequence. The set of points y with S(y) = S(x) is nowhere dense. 
The point x belongs to the closure of the set Vq . 

Proof. Let S(x) be periodic with the period In. Then the vector from x to T 2n x is a nonzero 
vector, otherwise x is a periodic point; call this vector v. This vector is determined by the sequence 
of vertices in the segment of S(x) of length 2n, so T 2nk x = x + kv. This means that the orbit of 
x is unbounded. The first statement follows. 

To prove the second one, let U be the largest connected open set consisting of points y with 
S(y) = S(x). Since T is area preserving and its orbits are bounded, there exists an n such that 
T 2n U Pi U is not empty. One can connect any two points of the set T 2n U U U by a path that does 
not intersect Vq, hence they all are encoded by the same sequence. This contradicts the maximality 
of U. 

The third statement readily follows. Any neighbourhood W of x contains a point y with 
S(y) ^ S(x). Let n be the first time y and x reflect in distinct vertices of the dual billiard polygon. 
Then T n ~ 1 x and T n ~ 1 y are separated by a segment from Vq. Therefore Vo intersects W. Q.E.D. 

We now turn to a more detailed study of the simplest quasirational polygon that fails to be a 
lattice one, namely, an (affine) regular pentagon. We follow [Ta 1,2]. 

Consider the following computer picture. The white regions, which are actually regular pen- 
tagons or decagons, consist of periodic orbits. The black "web" is the set of points with infinite 
orbits or, equivalently, the closure of the set Vo- Distinct infinite orbits constitute "web necklaces" 
around the pentagon. Notice self-similarity of the web clearly seen on a blow-up. 

fig. 106 

We shall study the dual billiard transformation inside its smallest invariant region shown in 
the next figure. The situation outside of this region is similar, due to the periodicity involved in 
the proof of Theorem 3. 

fig. 107 

Theorem 6. The Hausdorff dimension of the set of points with infinite orbits equals 

log 6 
log(v / 5 + 2)' 

Each orbit is dense in (that is, the system (V^T) is minimal). A connected component of 
the set Vf of periodic points is either a regular decagon or a regular pentagon. The periods of 
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even-periodic (that is, generic) points equal 

y(8- 6"" 1 + (-1)"); n=l,2,... 

in the former case, and 

^( 6 » + (-l)«-i): n=l,2,... 

in the latter one. 

Proof. The exterior of the pentagon is the union of its five identical exterior angles; identify 
them by the rotation about the center of the pentagon through 2ir/5. Let V be the part of 
one of these five exterior angles inside the invariant set under consideration, and let T be the 
transformation of V induced by the dual billiard map. 

fig. 108 

Denote by A and B the points on the bisector ON such that the clockwise rotation about 
A through 37r/5 sends the point K to M, and the clockwise rotation about B through ir/5 sends 
M to K. Denote these rotations by a and b, respectively. The first observation is made by an 
inspection of the figure 108: T is a piecewise isometry whose restriction to the triangle OKL is a, 
and whose restriction to the triangle LMN is b. 

Assign to a point its itinerary which is a word in the characters a and b that encodes which 
of the two rotations are applied to the point under consecutive iterations of T. Two points with 
the same itineraries belong to the same connected component of the set V — Vq (see the proof of 
Lemma 5). 

Denote by D the composition of the dilation, centered at O, that takes B to A, with the 
reflection in the line AB. Then D decreases distances by the factor of A = y/5 — 2, and it sends 
N to Ni, M to K\, K to Mi, etc. Inspecting figure 108 one makes a crucial observation: if 
x € AOKL then DT(x) = T 7 D(x); and if y € ALMN then DT(y) = T 3 D(y). More specifically, 

Da(x) = aababaaD(x); Db(y) = aaaD(y). 

Consider figure 108 again. One sees two periodic domains in it: a big T-invariant regular decagon 
and two big regular pentagons interchanged by T. Apply D and T in all orders to these polygons 
to obtain new regular decagons and pentagons, which are also periodic due to the above made 
observation. Given a periodic orbit a define its rank to be the maximal n for which there exists 
x 6 a such that D~ n (x) is still inside the quadrilateral OKNM. Periodic orbits of rank zero 
constitute the above mentioned big regular decagon and the two big regular pentagons. 

We claim that all periodic orbits are obtained from those of rank zero by applying compositions 
of D and T. Indeed, if a is periodic of rank n let x be its point inside OK 1 N 1 Mi such that 
D~ n (x) G OKNM. Then, again by the above observation, D~ 1 (x) is periodic, and the rank of its 
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orbit is n — 1 . Then one proceeds inductively. One also concludes that T acts transitively on the 
set of periodic decagons of a fixed rank, as well as on that of periodic pentagons of a fixed rank. 

It follows that the sets Vf and are self-similar. Namely, let W\ and W2 be the parts 
of Vqo in AOKL and ALMN, respectively. W2 consists of two parts A-homothetic to W\\ and 
W\ consists of five parts A-homothetic to W\ and three parts A-homothetic to W2 (see figure 108 
again). We apply the general techniques from [Fa] to find the Hausdorff dimension. Let d be the 
equal Hausdorff dimensions of W\ and W2, and W\ and W2 be their d- volumes. Then 

Wl = 5X d wi + 3X d w 2 , w 2 = 2X d w 1 . 

Hence X d = 1/6 and 

log 6 
~ log( v / 5 + 2)' 

To show that each T-orbit is dense in let x, y € Voo and e > 0. We want to show that 
T n (x) is e-close to y for some n. Choose N be big enough so that there exist periodic polygons X 
and Y of the same rank and with the same number of sides (i.e. two decagons or two pentagons), 
such that 

diamX < e/2, diamY < e/2, dist(x,X) < e/2, dist(y,Y) < e/2. 

Since T acts transitively on periodic polygons of the same rank and kind there exists an n such 
that T n X = Y, and the itineraries of X and x coincide up to the n-th place. Then T n (x) is e-close 
to y. 

Finally we compute the periods. Let a n and /3 n be the number of a's and 6's in the itinerary of 
a periodic polygon of rank n (we consider this itinerary as a finite sequence). The above discussed 
self-similarity implies: 

«n+i = 5a n + 3(3 n , f3 n+ i = 2a n . 

For the decagons a\ = 1 and (3\ = 0, and for the pentagons a\ = (3\ = 1. Solving the linear 
recurrences one finds: 

«n = \(6 n + ("I)"" 1 ), Pn = ^(e- 1 + (-1)"), 

for the decagons, and 

a n = i(9-6"- 1 + 2-(-l)"), n = I( 3 .6"- 1 +4-(-l)' 1 - 1 ) 
for the pentagons. Thus in the former case 

Pn = a n + (3 n = ^(8-6 n - 1 + (-l) n ), 

and in the latter one 

q n = a n + f3 n = ^(6 n + (-l) n - 1 ). 
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The first numbers are always odd, so the return map for a periodic decagon is the reflection in its 
center; thus the period of a generic point equals 2p n . The return map for a periodic pentagon is 
the identity, so the period of every point is q n . These numbers are the periods for the map induced 
by the dual billiard transformation in one of the exterior angles of the pentagon. Thus the periods 
for the dual billiard map are five times as big. Q.E.D. 

Itineraries of points provide a symbolic description of the dynamics on the set V^. Namely 
one considers the closure of the set of sequences obtained from 

...aababaa aababaa aaa aababaa aaa aababaa aababaa... 

in the space of all sequences in the symbols a and b. This sequence is invariant under the substi- 
tution 

D(a) = aababaa, D{b) = aaa, 

and T acts as a shift: T(c n ) = (c' n ) with c' n = c n+ \. The infinite sequence is aperiodic although 
the ratio of the symbols a's to the symbols 6's in it equals 3. 

Regular n-gons with n > 7 have not been studied yet, but the situation appear to be quali- 
tatively the same. The next figure shows infinite orbits of the dual billiard map for some regular 
n-gons (n = 8,9,10,11,12,17,20,25). 

fig. 109 

4.5 Higher-Dimensional Dual Billiards 

This section is concerned with higher-dimensional dual billiards. We follow [Ta 1,2]. The 
definition of the billiard transformation in higher-dimensional setting does not cause any trouble - 
it goes exactly along the lines of the two-dimensional case. Consider a smooth closed hypersurface 
M in a linear space. How does one define the dual billiard transformation associated with Ml The 
obvious difficulty is that, given a point in space, there are too many tangent lines through it to M. 

As a motivation consider again higher-dimensional billiards (compare to Section 1.5). Let 
pn £- jjn+i kg a hypersurface. Reflection in T is the billiard transformation of the set of rays 
N 2n in R n+1 . The set of rays is a symplectic manifold and the billiard transformation preserves 
the symplectic structure. To V a hypersurface S C N corresponds that consists of rays tangent 
to r. A characteristic curve on S consists of rays tangent to T along a geodesic line on it. Let a 
ray r hit V at x, and let r\ be the reflected ray. The rays r and r\ generate a 2-plane, and the 
rays, that lie in this plane and pass through x, form a line in N. This line is tangent to S and 
has the characteristic direction at the point of tangency. Also we have the condition: the angle of 
incidence equals the angle of reflection. 

A loose translation to the language of dual billiards yields the following definition. 

Definition. Let M 2ra_1 C R 2ra be a smooth closed hypersurface in linear symplectic space. 
Two points x and y are in the dual billiard relation T with respect to M if the line xy is tangent 
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to M, has the characteristic direction at the point of tangency, and the segment xy is bisected by 
the tangency point. 

Let us remark that this definition still makes sense when M is a front, that is a singular 
hypersurface with a well-defined tangent hyperplane at each point. Identify R 2n with C n . Then 
the characteristic direction at a point of M is obtained from the normal one by multiplying by 
yj— 1. Thus x and y are in the dual billiard relation if there exists a point z £ M such that 
zx = y/—l N z , zy = — yj — 1 N z for some normal vector N z to M at z. One can slightly generalize. 
Let a be an angle. Define the relation T a : x and y are in T a if there exists z € M such that 

zx = exp(v / — Ta) N z , zy = exp(— v 7 — la) iV z 

for some normal vector N z . The dual billiard relation T is a particular case of T a for a = it/ 2. 

Theorem 1. T and T a are symplectic relations. 

Proof. Let ui be the linear symplectic structure in R 2n , and let G C R 2n x R 2n be the 
graph of T. We want to show that G is a Lagrangian submanifold, the symplectic structure in 
the product space being oj\ 6^2, where u>i are the linear symplectic structures in the factors. Let 
x,y be Darboux coordinates in the first copy of R 2n , so that u\ = dx A dy, and x, y be Darboux 
coordinates in the second one. Consider the cotangent bundle T*R 2n with its canonical symplectic 
structure; let 91,92 be space and pi,P2 be momentum coordinates. Then the linear map 

x + x y + V _ _ 

9i = —2~i q2 = ~2~' Pl=y ~ y > P2=x-x 

is a symplectomorphism of T*R 2n to (R 2n x R 2n , oj x Q uj 2 ). 
The graph G C T*R 2n is contained in the set 

{(q,p) G T*R 2n | q € M, p € Ann T q M}. 

This set is the conormal bundle of M C R 2n , and therefore a Lagrangian submanifold. 

Similarly, the graph G a of T Q in T*R 2n is contained in the set of (q,p) € T*R 2n such that 

q is the end-point of a normal vector N to M, p is the covector cot a < N, ■ > at q. 

This set is the graph of the differential of the following function in R 2ri : 

f(q) = cot a x distance 2 (q, M), 

and therefore is a Lagrangian submanifold. Q.E.D. 

Notice that the above considered sets are bigger than the graphs of T (or T a ), namely they 
are the unions of the graphs of T and of T_i (or of T a and of T~ v ). 

So far we have defined the dual billiard relation T and included it into a family of relations T a . 
To make it into a map we need M to be strictly convex (just as in the case of the plane). Notice 
that an orientation of M induces an orientation of its characteristic lines, which will be refered to 
as characteristic rays. 
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Lemma 2. Let M be closed and strictly convex. Then for any point x outside of M there exists 
a unique point y £ M such that yx is the characteristic ray to M at y. Likewise, given an 
angle < a < tt/2, for any point x outside of M there exists a unique point y £ M such that 
yx = exp( v / — T a) N y for some outward normal vector N y to M at y. 

Proof. Let N y be the unit outward normal vector to M at y. Consider the map F a from 
M x [0, oo) to the exterior of M: 

F a (y,t) = y + t exp( v /T T a) N y . 

This map has degree one so it is onto. Assume F a (y 1 ,ti) = F a (y 2 ,t 2 ). Let TVj = ti N y . and 
Ti = v 7 — T Ni, i = 1,2. Then y 1 + cos a N 1 + sin a T x = y 2 + cos a N 2 + sin a T 2 , and thus 
V2-yi= cos a (TVi - iV 2 ) + sin a (Ti - T 2 ). 

fig. 110 

Notice that u>(Ni,Ti) > 0. Let v be an outward vector at yi with normal component N. Then 
uj(v,Ti) = uj(N,Ti) since Ti belongs to the kernel of u, restricted to the tangent hyperplane at yi. 
Thus oo(v,Ti) > 0. Consider the vector v = y 2 — y\. At the point y 2 it has the outward direction, 
so uj(v,T 2 ) > 0. Likewise u>(v,Ti) < 0. Subtract the latter inequality from the former to get: 
lo(v,T 2 —T\) > 0. Substitute the above obtained expression for v and use the fact that u is skew- 
symmetric: cos a x u;(N 2 — N\,T 2 — Ti) < 0. If a = tt/2, which is the case with the dual billiard 
relation, this is a contradiction. Otherwise uj{N 2 — N\, T 2 — 7\) < 0, which is a contradiction again 
since u>(u, v 7 — T u) > for any vector u. Q.E.D. 

Thus in the presence of symplectic structure one defines "the tangent line" to a hypersurface at 
a point as the characteristic line, and the characteristic half-lines to a strictly convex hypersurface 
foliate its exterior, just as in the plane. One defines the dual billiard transformation of the exterior 
of M: given a point x outside of M, find the unique point y £ M such that yx is the characteristic 
ray to M at y, and define T{x) to be the reflection of x in y. Likewise one defines the transformation 
T a . Notice that T commutes with linear symplectic transformations, while T a is invariant under a 
smaller group of motions. 

The next problem to address concerns periodic points of the dual billiard transformation T. 

Theorem 3. Let M be a smooth closed strictly convex hypersurface in linear symplectic space, 
and T the corresponding dual billiard transformation. For any odd prime k there exist k-periodic 
orbits ofT. 

Proof. We use the method developed in [Gi]. Consider the product (R 2n ) fc x (R 2n ) fe with 
the symplectic structure lo\ Q lo 2 . Let z\, Zk be coordidinates in the first, and Zi, ...,~z~k in the 
second factor (so that each z and z is a vector in R 2n ). Consider two submanifolds: 

G k = {z- = T ±1 z l } and C = {zl = z i+1 ] 



97 



where i = l,...,k and k + 1 is understood to be equal to 1. The intersection G k n C consists of 
/c-periodic orbits of the relation T, that is of chains zi,...,z k such that z i+1 = 

As in Theorem 1, (R 2n ) fc x (R 2n ) fc is symplectomorphic to (T*R 2n ) fc . Let qi,...,qk be space 
and p\,...,pk be momenta coordinates. Then G k is the conormal bundle to M k = M x ... x 
M (k times) C (R 2n ) fe . C is a linear subspace in (T*R 2n ) fc , and a direct computation shows that 
for odd k it is the graph of the differential of the following quadratic function: 

*(?!,..., 9fc) = u(qi,qj), 

i<j 

where the points qi are treated as vectors in linear space. 

Thus the points of G k C\C are critical points of the restriction of $ on M k . Since M k is compact 
this intersection is not void. By Morse theory there are at least k + 1 points in this intersection, 
and at least 2 fc for M in general position. However some of the critical points correspond to "fake' 
orbits with Zi = Tzi 1 , z i+1 = T~ x Zi for some i. If q = (qi,...,qk) is the corresponding critical 
point of then q^i = qi. Consider a tangent vector £ to M k at q all of whose components vanish, 
except for (i — l)-th and i-th, and these two are equal to a non-zero tangent vector to M at qi. 
Skew-symmetry of uj implies that d 2 $(£) = 0, so the second differential of $ is not positive at 
"fake' periodic orbits. It follows that maxima (or minima) of <& correspond to /c-periodic orbits of 
T , maybe, multiple ones. Since k is prime this multiplicity equals one. Q.E.D. 

We would like to make several comments on this theorem. First, it seems plausible that the 
dual billiard transformation has /c-periodic orbits for every k > 3. There are two difficulties in 
proving it. On the one hand, if k is even, the above generating finction for the linear subspace 
C does not work because the projection of C on the diagonal has a kernel. This difficulty can be 
overcome by applying the symplectic reduction or by a small perturbation of the projection, as in 
[Gi]. On the other hand, if k is not prime one should be able to distinguish between "faithful" 
orbits and multiple orbits of smaller periods. This is a common problem in Morse theory. Another 
question of interest is whether one can improve the result even for prime k: are there more than 
just one periodic orbit? 

Secondly, the functional $ in the plane case is different from the area functional on circum- 
scribed polygons. For example, 5-periodic points correspond to extrema of $ shown in the next 
figure. Similarly in the billiard problem there exists a functional other than perimeter, responsible 
for periodic trajectories, which might prove useful. 

fig. Ill 

Another remark concerns the relation between periodic orbits of the dual billiard transforma- 
tion and closed characteristic lines on M. In a sense, the dual billiard map is a discretization of 
the characteristic flow on M. It is known that M always possesses a closed characteristic (see, 
e.g. [Sik]), and it is conjectured that there are at least n of them (2n being the dimension of 
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the ambient space). The question is whether one can use periodic trajectories of the dual billiard 
map to approximate closed characteristics. More specifically, does there exist a sequence of such 
periodic trajectories with the periods going to infinity that accumulates to any closed character- 
istic? Closed characteristics are extrema of the symplectic area functional on closed curves in M 
(that is, the integral of the Liouville 1-form over a curve or the integral of the symplectic form 
over a film, bounded by the curve). Let 7 : [0, 1] — > R 2n be a smooth curve; partition it by the 
points qi = / y(i/(2N + 1), i = 0, ...,27V. One expects the function <&(qo, (few) to approximate 
the symplectic area of a film, bounded by 7. 

Lemma 4. $(qo, <?2a0 — ► Jq w(j(t), j'(t)) dt as N — > 00. 

The proof is a straightforward estimation, and we omit it. 

As in the plane case, the dual billiard transformation associated to an ellipsoid is integrable. 
Up to a linear symplectic map, any ellipsoid is given by the equation a % ( x f + Vi) = 1 where 
(xi,yi) are Darboux coordinates. Let X{ = r^cosc^, = rj sin aj. In these polar coordinates the 
dual billiard map acts as follows: T(ri,ai) = (rj,/3j) where = a, + tan _1 (ai/i) and t is the root 
of the equation 



Thus the radii 77 are integrals. Are there other integrable dual billiards in higher dimensions? 

An interesting question is whether orbits of the dual billiard transformation may escape to 
infinity or "fall" on the hypersurface M. In the plane, KAM theory provided invariant curves which 
prohibit both events. In higher-dimensional situation, even if KAM theory is applicable, it would 
provide invariant tori which no more separate the space, and therefore do not ensure stability. 
Moreover, for a generic hypersurface M the dual billiard map is not close to an integrable one 
either near M or at infinity. At a vicinity of M its trajectories are approximated by the leaves of 
the characteristic foliation on M, and at infinity - by those of the characteristic foliation of another 
hypersurface, constructed from M by symmetrization and polar dualization as in Section 4.2 (we 
will not work it out in detail here). 

In conclusion we mention a higher-dimensional analog of polygonal dual billiards, namely, dual 
billiards associated to convex polyhedra. Identify R 2n with C n and let J be the multiplication by 



V— 1. Let P be the boundary of a convex polyhedron. Given a point x G P let C x be its dual 

cone, that is the convex hull of the outward normal vectors to (2n — l)-dimensional faces of P that 
contain x. If x belongs to the interior of a fe-dimensional face then C x is (2n — /c)-dimensional. The 
exterior of P is the union of these dual cones. Rotate each cone C x by J about x; an argument 
similar to that of Lemma 2, shows that the union of JC X , x 6 P, covers the exterior of P. 

Consider a face F such that the restriction of uj to it is degenerate. Then the intersection of 
the rotated space JF (about one of its points) with the normal complement of F (at this point) 
is not trivial. Hence dim (Li xe p JC X ) < 2n, and this union is contained in the closures of other 
cones. We ignore such faces, in particular, odd-dimensional ones. Consider the prisms U x€ f JC x 
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for the faces F such that the restriction of to to F is nondegenerate. Their interiors are disjoint 
and their union V over all such faces covers the exterior of P except for a codimension one set. We 
define the dual billiard transformation of V: given a point x 6 V, find the unique point y G P such 
that x 6 JCy, and reflect x in y. Therefore the dual billiard map can be thought of as reflecting in 
even-dimensional faces of a convex polyhedron in linear symplectic space. The study of this map 
is an interesting problem. In particular, for which polyhedra are all the orbits bounded or finite? 
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5. Hyperbolic Billiards 



The last chapter concerns chaotic billiards. We start with a brief introduction to hyperbolicity; 
we consider two examples: a torus automorpism and the geodesic flow on a surface of constant 
negative curvature. Section 2 is devoted to dispersing billiards; the main mechanism of ergodicity 
is the divergence of nearby trajectories. Section 3 deals with chaotic billiards with convex arcs. 
We start with the famous "stadium" and proceed to several sufficient geometrical conditions for 
hyperbolicity. The last section contains a discussion on Bolzmann's Hypothesis, an estimate of 
the growth rate of periodic trajectories in dispersing billiards and an application of Poincare's 
recurrence theorem to the billiard under a curve lower-asymptotic to the horizontal axis. 
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5.1 Introducing Hyperbolicity 

This section is a very sketchy introduction to hyperbolic dynamics, whose particular case - 
hyperbolic billiards - is the topic of this chapter. Instead of formulating results in their generality 
we look at two examples (see [Si 1, Ni, Man]) , in which the situation appears particularly clear, 
and which serve as models for numerous more involved results concerning hyperbolic billiards. 

Let A be an automorphism of a two-dimensional torus T 2 = R 2 /Z 2 . Lifted to R 2 , the linear 
transformation A is given by a 2 x 2 matrix. To act on the torus it must preserve the integer 
lattice, so all the entries are integers; and being a diffeomorphism, the determinant equals ±1. 
Assume that A has real eigenvalues Ai and A2 with |Ai| > 1, | A2 1 < 1. Denote the corresponding 
eigenspaces by E u and E s ("u" for unstable and "s" for stable). 

Observe that the slopes of both spaces are irrational. Indeed, otherwise this space will project 
to the torus as a closed circle. The diffeomorphism A, restricted to this circle, would be either an 
expansion by |Ai| or a contraction by | A2 1 , which is impossible since a circle is compact. Thus the 
projections of E u and E s to the torus are dense immersed lines. By a parallel translation of the 
eigenspaces in R 2 and then projecting to the torus one obtains two transversal foliations W u and 
W s therein, invariant under A. 

fig. 112 

Equivalently, the leaf of W s through a point x consists of points y, asymptotic to x; 
W s = {y\ (list {A n {x), A n {y)) ^ as n -» 00}. 

Likewise, 

W u = {y\ dist {A n {x),A n {y)) ^ as n -» -00}. 

We claim that periodic points of A are dense in the torus. Indeed, any point with rational 
coordinates is periodic. Let d be the least common denominator of the two coordinates of a 
rational point x. Consider the entries of the matrix A modulo d. Since GL(2, Zj) is a finite group 
A n = Id mod d for some n. Hence A n {x) — x 6 Z 2 , which means that x is a periodic point. 

Associate a partition of the torus into two parallelograms with A. To fix ideas, let 



so that Ai = ^fi, A 2 = 
fig. 113 

The edges of the two rectangles lie on the expanding and contracting eigenlines of A. The 
image of the first rectangle is contained in the second one, while the image of the second one 
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intersects both. One can encode the orbit of a point x by a sequence of symbols 1 and 2, according 
as A n (x) belongs to the first or to the second rectangle. In this encoding no two consecutive l's 
appear; otherwise any combination of the symbols is possible (we ignore the ambiguity over which 
symbol to attach to a point on a side of a rectangle). In particular, one easily constructs a sequence 
that contains all blocks of l's and 2's with no two consecutive l's; to this sequence a dense orbit of 
A corresponds. This symbolic description also makes it possible to compute the number of periodic 
orbits of period n, which amounts to counting the number of n-length segments of l's and 2's with 
no two consecutive l's. This number grows exponentially in n. 

This partition of the torus and the corresponding symbolism is a particular case of a general 
technique, introduced by Adler and Weiss and by Sinai in the 60-s, called the method of Markov 
partitions (see [Bow, Si ?]). 

Our second example is the geodesic flow on a closed surface of constant negative curvature. 
The universal cover of such a surface is the hyperbolic plane, which we consider in Poincare's 
upper half-plane model. The metric is given by (dx 2 + dy 2 )/y 2 , and straight lines are represented 
by circles or straight lines perpendicular to the x-axis, which is the line at infinity. Consider a one- 
parameter family of asymptotic lines passing through one point at infinity. Applying a hyperbolic 
motion we may assume that it consists of parallel vertical lines. 



Consider two points that move along two asymptotic geodesies with unit velocities, starting 
at height 1. The distance between the points along horizontal line at time t equals 



Hence this distance decreases as e~*. 

Horizontal lines are perpendicular to the family of vertical asymptotic geodesies. In general, 
curves perpendicular to asymptotic geodesies are circles, that are tangent to the x-axis at a fixed 
point. They are called horocycles. Given a unit tangent vector v in the hyperbolic plane there 
are two horocycles, perpendicular to it, which pass through its foot point. Extend v to a unit 
normal vector fields along these horocycles; these framed horocycles are two curves in the unit 
tangent bundle of the hyperbolic plane, which pass through v and which are transversal to the 
geodesic flow. Thus the unit tangent bundle carries two one-dimensional horocyclic foliations. It 
follows from the above considerations that the geodesic flow exponentially contracts the leaves of 
one foliation and exponentially expands the leaves of the other (this can be also deduced from the 
formulas found in Section 3.5). 



Projecting to the unit tangent bundle of the compact surface one obtains the stable and 



fig. 114 




fig. 115 
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unstable one-dimensional foliations W s and W u therein. These foliations are analogous to the 
ones from the previous example. The existence of the two foliations makes it possible to prove the 
following theorem due to Hopf and Hedlund. 

Theorem 1. The geodesic Bow on a compact surface of constant negative curvature is ergodic. 

Proof. Let g t be the geodesic flow. Given a continuous function f(x), by Birkhoff's ergodic 
theorem 

km - / f(g t x) =U(x), lim - / f(g t x) = J-{x) 

almost everywhere, and f+(x) = f-(x) on a set of full measure. One wants to prove that f±(x) 
are almost everywhere constant. It is enough to prove this locally, that is in a neighbourhood of a 
point. 

Since / is uniformly continuous and since points of a leaf of the stable foliation are asymptotic 
in positive time with respect to the flow, f+(x) exists and has a constant value on such a leaf, 
provided it exists at one of its points. Likewise, /_ (x) is constant on a leaf of the unstable foliation. 
Since f+(x) is ^-invariant, it is constant on a two-dimensional surface which is the union of the 
images of a stable curve under the flow. One can choose a stable curve so that f+(x) = f-(x) 
almost everywhere on it. Then f+(x) = f-(x) = const on a subset U of full measure of the above 
constructed surface. Finally, the union of unstable curves through the points of U has full measure 
and f+(x) = f-(x) = const thein. 

Said otherwise, two points belong to the same ergodic component if they can be connected by 
a broken line that consists of stable curves, unstable curves and trajectories of the geodesic flow 
(such broken lines are called Hopf chains). Any two points of a neighbourhood in the unit tangent 
bundle can be connected by a Hopf chain, so the geodesic flow is ergodic. Q.E.D. 

The study of geodesic flows on Riemannian manifolds of negative curvature goes back to 
Hadamard, Morse, Hopf and Hedlund. The situation with a compact manifold of negative curvature 
is similar to the one with a surface of constant negative curvature: almost all trajectories of the 
geodesic flow are dense; the flow is ergodic and mixing; periodic trajectories are dense, and the 
number of closed geodesies of length not greater than I is finite and grows exponentially with I (see 
[An]). 

Thus the dynamics in the two examples enjoys similar stochastic properties. The basic reason is 
the existence of a decomposition of the tangent space at each point into contracting and expanding 
subspaces, invariant under the dynamics. Let F be a measure-preserving diffeomorphism of a 
compact Riemannian manifold M (the measure being equivalent to the Riemannian volume). By 
the Oseledets multiplicative ergodic theorem ([Os]) at almost every point x G M there exists a 
DF-invariant measurable decomposition T X M = ®Hi{x) such that 

lim - log \\DFZ(v)\\ = X i(x) 

uniformly over unit vectors v 6 Hi(x). The numbers Xi( x ) are called Lyapunov exponents. The 
reader recognizes the spaces Hi(x) as the stable and unstable eigenspaces of a torus automorphism, 
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and the Lyapunov exponents as the corresponding eigenvalues. The diffeomorphism F is called 
hyperbolic if none of the Lyapunov exponents vanishes (similar definitions are given for vector 
fields, in which case the direction of the field necessarily contributes a zero exponent). 

A general theory of hyperbolic systems was constructed by Ya. Pesin (see [Pes 1,2; Pol]). A 
particular result of this theory is the following Pesin's formula that expresses metric entropy in 
terms of the Lyapunov exponents. 

Theorem 2. 

K F ) = I (Y\ Xi(x) dimHi (x)) dfi, 

J M 

where the summation is taken over positive Lyapunov exponents. 
Another expression for the entropy is 

h(F) = [ log det {DF%) d n, 

where DF U is the restriction of the differential of F on the unstable subspace E u = ®Hi(x) over 
Xi(x) > 0. 

Since the billiard transformation has singularities Pesin's theory is not directly applicable to 
it. A relevant generalization of this theory to hyperbolic systems with singularities is contained in 
the book by A. Katok and J.-M. Strelcyn [K-S]. 

5.2 Dispersing and Semi-Dispersing Billiards 

Before we start our disscusion of hyperbolic billiards let us refer to three surveys of this topic: 
[Bu 1, Si 3, G-C]. 

The first class of stochastic billiards had been discovered by Ya. Sinai (see [Si 4]). These 
billiards are bounded by piecewise smooth curves whose smooth components are strictly convex 
inwards, and which intersect transversally. The following figure shows three examples: a simply 
connected plane billiard, a plane billiard with a hole and a billiard on a torus with a hole. 

fig. 116 

A parallel beam of rays, after a reflection in a side, becomes dispersing. For this reason 
these billiards are called dispersing (they are also known as Sinai's billiards). Each consecutive 
reflection forces the beam to further diverge. Reversing the time direction one obtains a beam that 
exponentially converges. This situation is typical for hyperbolic dynamics. Dispersing billiards also 
exist in higher-dimensional spaces; they are bounded by hypersurfaces strictly convex inwards. 

fig. 117 

An intuitive explanation why one may expect chaotic dynamics in dispersing billiards was 
proposed by V. Arnold (see [Ar 2]). Consider the billiard on a torus with a hole. Its double 
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(see Section 1.1) is a sphere with two handles with a Riemannian metric of negative curvature 
(supported in a neighbourhood of the hole). The billiard flow is the limit of the geodesic flow on 
this surface, which enjoys stochastic properties; thus one expects the billiard flow to be stochastic 
as well. 

Recently I. Babenko ([Bab 2]) analyzed this construction rigorously. At a neighbourhood of 
the hole, the metric of the surface is (1 + \u\ k(s)) 2 ds 2 + du 2 , where s is the length parameter on 
the boundary curve, k(s) is its curvature, and \u\ is the distance to the curve. The difficulty arises 
from the fact that this metric is not smooth along the curve. One lifts the billiard flow to the 
universal cover which is the hyperbolic plane, and studies "reachable sets" on the circle at infinity, 
that is sets which can be reached from a given set by moving along billiard trajectories. These sets 
are Cantor-like, and the topological entropy of the original billiard is estimated in terms of their 
Hausdorff dimensions. 

Back to dispersing billiards. Sinai constructed local stable and unstable curves through almost 
all points of the phase space, i.e., curves whose points are exponentially asymptotic for positive or 
negative time. We remark that these curves cannot be too long, due to singularities. Namely a 
stable or unstable curve breaks when the trajectory of one of its points hits a corner or becomes 
tangent to the boundary. 



Consider a one-parameter family of rays that undergo reflections at the same boundary compo- 
nents. Let 7 be a curve perpendicular to the rays (a wave front). Being framed by the unit vectors 
of the rays it determines a curve 7 in the unit tangent bundle of the plane. We will investigate how 
the curvature of 7 changes under the billiard flow g t . For 2; € 7 denote by x( x ) the curvature of 7 
at x. Consider the evolution of the front until the first reflection in the boundary. Let t\ be the 
distance to the boundary from x along the ray through it, and let Xi be the curvature of gt 1 (j) a t 
the point g tl (x) immediately before reflection. Approximating 7 by its osculating circle at x one 
finds: 



Consider now how the curvature of a front changes under reflection. Let k\ be the (positive) 
curvature of the boundary at the point where the ray through x hits it, 4>\ be the angle between 
this ray and the boundary, and xt be the curvature of the front immediately after reflection. In 
this setting the mirror equation from Lemma 2.4.2 reads as follows: 



fig. 118 
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Xi - xt = 



2h 



COS</>i 
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Combining the two equations we find: 
Iterating reflections we get 

X(x) = 



2h + 
+ Xi 



COS 01 



2k! 

+ 



COS 01 

h + 



2k 2 



COS <p2 

etc. 

Suppose that the rays constitute a local stable curve, so that the rays become asymptotically 
parallel. Then xf ~~ ^ an d we arrive at a continued fraction expression for the curvature of its 
front: 

X(x) = l — 

h + 



2A- 1 



COS( 



2k 2 1 



COS 02 h + 

All terms here are positive, and the criterion for convergence of such a continued fraction is that 

J>i + ~^-) = oo 

^ COS (pi 

(see [Kh]). This condition holds because the trajectory of x makes a finite number of reflections at 
any finite time interval. Notice that % is a function on the unit tangent bundle. We remark that 
the continued fraction essentially gives a solution of the Jacobi equation for a geodesic flow. 

Thus a local stable curve 7 s through a point of the phase space is uniquely determined by the 
curvature of 7, given by the above continued fraction. Reversing the framing of 7 one constructs an 
unstable curve j u . A similar argument applies to the higher-dimensional case in which an operator- 
valued continued fraction gives the second quadratic form B(x) of a local stable manifold. 

A detailed analysis shows that local stable and unstable curves pass through almost every 
point of the phase space. Thus dispersing billiards are hyperbolic dynamical systems. 

To evaluate the metric entropy of a dispersing billiard one uses the second formula for entropy 
from the previous section. Let V be the phase space consisting of unit inward vectors with the 
footpoints on the boundary, with the coordinates (a,t) introduced in Section 1.2; T : V — * V be 
the billiard map, and v = sin a da dt - the invariant measure. Given a unit tangent vector x 6 V 
denote by t(x) the length of its free path until it hits the boundary. 
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Theorem 1. 



h(T) = [ log \l + t(x) X (x)\ dv{x). 
Jv 



This formula, discovered by Sinai ([Si 4]), holds not only for dispersing, but also for other 
hyperbolic billiards which we discuss in the next section (see [Ch 1 , Ch-M] ) . The entropy of the 
billiard flow in the unit tangent bundle M is 

Hdt) = / X(x) dfi(x), 

JM 

where p is the Louiville measure. These formulas also have higher-dimensional analogs: 

h(T) = / log del (I + t(x) B(x)) du(x), 
Jv 

Hat) = tr B(x) dp(x). 

J M 

Proof. We need to evaluate the rate of expansion of the unstable direction under the billiard 
transformation in the Euclidean metric dl 2 = da 2 + dt 2 . What we know is how the curvature of 
the front corresponding to an unstable curve changes during a free path: 

= t or *L = l + tx. 

X Xi X 



fig. 120 

Let dp be the length form of the front; since curvature is inverse proportional to length 

dpiTx) 



dp(x) 



l + t(x) x(x). 



In terms of the (a,t) coordinates dp = sin a dt (see the figure). Let 

j( \ _ dl(x) 1 I da 2 
dp{x) sin a V dt 2 

on an unstable curve. Then 

dl{Tx) dp{Tx) J(Tx 



dl(x) dp(x) J(x) 
Since the measure v is T-invariant 



(l + t(x) X (x)) 



J{Tx) 



and therefore 



/ log 

J v 



/ log J(tx) du(x) = / log J(x) dv{x), 
Jv Jv 

"' ! ' " ' dv(x) = [ log 1 1 + t(x) x(x)\ dv{x). 
Jv 



dl(x) 
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Here one makes use of integrability of the function 



.., da 2 . 

which is proved in [Ch 1]. Q.E.D. 

To apply the method of Hopf chains to dispersing billiards it is necessary to control the size 
of local stable and unstable manifolds. This was achieved by Sinai ([Si 4]), and the result is known 
as the Fundamental Theorem of the theory of hyperbolic billiards. We formulate it in a slightly 
weakened form. 

Theorem 2. If the trajectory of a phase point of a dispersing billiard is never tangent to the 
boundary nor hits a corner then, given a local unstable curve of length I in a neighbourhood of this 
point, the probability to find a local stable curve of length CI, where C is an arbitrary constant, 
tends to 1 as the radius of the neighbourhood goes to zero. The statement holds true if one 
interchanges the words "stable" and "unstable". 

A consequence of the Fundamental Theorem is the following result. 

Theorem 3. A dispersing billiard is ergodic. 

Other proofs and modifications of these theorems are found in [Bu-Si 1, Si 5, Ch-Si]. It 
follows from Theorem 3 and hyperbolicity that the billiard transformation is mixing and even has 
K-property. A two-dimensional dispersing billiard is also isomorphic to a Bernoulli shift ([G-O]). 

Some of these results hold for semi-dispersing billiards, which are billiards whose boundary 
components have non-negative inward curvature. 



fig. 121 



Straight components of the boundary are called neutral; unlike dispersing ones they do not 
change converging properties of incoming trajectories: after a reflection a parallel beam remains 
parallel. An example of a semi-dispersing billiard is a three-dimensional torus with two deleted 
non-parallel cylinders, studied in [KSS 1]; we will discuss its physical interpretation in Section 4. 

Stable and unstable manifolds for semi-dispersing billiards are constructed in [Ch 2]; unlike 
the case of dispersing billiards their dimensions may be less than one half of the dimension of the 
phase space. A version of the Fundamental Theorem for semi-dispersing billiards is given in [KSS 
2], see also [Ch 3]. 

As the example of a torus automorphism suggests, Markov partitions are very useful in the 
study of hyperbolic dynamics. Markov partitions for dispersing billiards were first constructed in 
[Bu-Si 2], and further improved in [B-C-S 1], see also [Kr-Tr] and [Ch 4]. The construction is 
much more delicate than that for torus automorphisms. An element of a Markov partition is a 
"rectangle" consisting of intersections of local stable curves through a subset of a given unstable 
curve and local unstable curves through a subset of a given stable curve. These sets are typically 
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Cantor-like, and the number of elements in a partition is necessarily countable, due to singularities 
of the billiard transformation. Using Markov partitions Bunimovich, Chernov and Sinai proved 
the following theorem. 

Theorem 4. Periodic orbits of a two-dimensional hyperbolic billiard are dense in the phase space. 
The number of periodic orbits of period not greater than N is bounded bellow by e CN for some 
constant C and sufficiently great N. 

The same authors also used Markov partitions to study strong statistical properties of two- 
dimensional hyperbolic billiards, such as the central limit theorem and time decay of correlation 
functions - see [Bu-Si 3, B-C-S 2]. 

5.3 Hyperbolic Billiards with Focusing Arcs 

Chaotic behavior in billiards discussed in the previous section was due to scattering of trajecto- 
ries upon reflection in the boundary. Quite a different chaos producing mechanism was discovered 
by L. Bunimovich in 1974 ([Bu 2]), who demonstrated that there exist convex ergodic billiards. His 
billiards are bounded by arcs of circles (focusing components of the boundary) and straight seg- 
ments (neutral components), with the condition that the circle, containing a focusing component, 
lies strictly inside the billiard table. Some examples are shown in the figure. 

fig. 122 

The first billiard is the famous "stadium" bounded by two half-circles and two segments; it is 
a C 1 -smooth curve. The last billiard does not belong to the described class, but the dynamics in it 
reduces to that in its double - the union of the domain with the one symmetric to it with respect 
to the segment; this double already satisfies the above condition. 

Since the pioneering work by Bunimovich the conditions on focusing components, that en- 
sure hyperbolic dynamics, had been relaxed in the works of many authors, and up to now several 
constructions of such billiards are known. In all these examples the billiard curves are at most 
C 1 -smooth. A convex ergodic billiard cannot be too smooth in view of Lazutkin's theorem (Section 
2.8); the least smoothness prohibiting ergodicity is not known. A numerical study of a transfor- 
mation of the billiard inside a circle to the one inside a stadium is found in [B-S]. 

A heuristic explanation of how focusing arcs may contribute to hyperbolicity goes as follows. 
Consider an incoming infinitesimal beam of parallel rays. After a reflection it focuses, and then, 
before the next reflection, it may defocus. If the time between focusing and the next reflection 
exceeds that between the previous reflection and focusing, that is if the beam spends more time 
diverging than converging, then close trajectories will diverge in the phase space, similarly to 
dispersing billiards. 

fig. 123 
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Bunimovich applied the technique involving continued fractions to billiards with focusing arcs. 
An obvious difficulty in proving that such continued fractions converge is that not all their terms 
are positive any more. Bunimovich proved the following result concerning the class of billiards 
described at the beginning of the section (as well as some other billiards with focusing arcs - see 
[Bu 2, 3, 4]). 

Theorem 1. The billiard transformation is ergodic and isomorphic to a Bernoulli shift. 

Here we describe an approach due to M. Wojtkowski ([Wo 2]). It was further developed by R. 
Markarian ([Mar 1, 2]), V. Donnay ([Don 2]); see also [Ch-M]. 

Let T be the billiard transformation of the phase space V consisting of unit inward tangent 
vectors with the footpoints on the boundary. Being a two-dimensional area preserving transforma- 
tion T has two Lyapunov exponents A+ > and A_ < with A_(x) = — A+(x). Let C(x), x € V 
be a measurable field of closed sectors in the tangent bundle of V, i.e. C(x) is a closed sector in 
T X V defined for almost all x and depending on x measurably. We say that T preserves C(x) if 
DT{C{x)) C C{Tx) almost everywhere; T strictly preserves C(x) if the inclusion is strict almost 
everywhere; and T eventually strictly preserves C(x) if T preserves C(x) and for almost every x 
there exists a positive integer n(x) such that DT n ( x \C(x)) is strictly inside C{T n ^ xS) x). 

The next statement is Wojtkowski's projective criterion for hyperbolicity that holds for piece- 
wise differentiable invertible area preserving mappings ([Wo 2, 3]). 

Theorem 2. If there exists a measurable Geld of sectors eventually strictly preserved by T then 
the Lyapunov exponent A+ is almost everywhere positive. 

To be able to apply this criterion to billiards we need to describe the action of the differential 
of the billiard transformation on the projectivized tangent bundle PTV. Let x = (i, v) G V, t is a 
point on a billiard curve 7, v - a unit vector, and let r be the corresponding ray. A tangent vector 
v € T X V determines an infinitesimal family of rays that includes r. Let /o be the signed distance 
from t to the point on r in which this family focuses in the linear approximation; the sign is chosen 
according to the direction of r. Since /o depends on the direction of v only, one may take it as a 
projective coordinate in PT X V. 

At each nonfiat point t of 7 consider the disc D(t) which is obtained from the osculating disc 
of 7 at t by the dilation, centered at t, with the coefficient 1/2. Denote by d the length of the 
segment of an incoming ray r inside D(t). Let k be the curvature of 7 at t, and 9 - the angle 
between 7 and r. Then 

sL~9 = {S9H k) \- 

fig. 124 

Let Tx = y, x = (t,v), y = (ii,t>i), and denote by L the distance between the points t and 
t\. Given a tangent vector v 6 T X V set v\ = DT(u), and let /1 be the corresponding projective 
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coordinate in PT y V, i.e. the signed distance from t\ to the point on n in which the outcoming 
infinitesimal family of rays focuses. Notice that /o — L is the signed distance from t\ to the focusing 
point on r. The mirror equation from Section 2.4 reads: 

1 1 2fci 1 1 , , 2 

fx L-fo sin 0i /i L-/ dx 

Thus 

f (/o - £) di 



2(/ -L) (s 5 n fcx) +di 

is the projective action of the differential of the billiard map. 

Now define a field of sectors C{x) in TV. For a focusing (convex outward) boundary component 
C{x) C T X V, x = (t,v) is defined by the condition that the focusing point of the respective 
infinitesimal family of rays lies in D(t). Equivalently, C is defined by the inequality < / < d. 
For a dispersing component C is defined by the condition that the focus of the infinitesimal family 
of rays lies on the exterior side of the boundary: — oo < / < 0. Neutral components are irrelevant: 
one can unfold a billiard trajectory that hits such a component and proceed to another copy of 
the billiard table (the method familiar from Chapter 3). In other terms, one deals with a smaller 
phase space of unit vectors over the nonflat boundary components, the transformation being the 
first return map of T. 
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What are the conditions on the billiard curve ensuring the preservation of the field of sectors 
C{x)l Consider the segment of a trajectory between two consecutive reflections in nonflat boundary 
components 7 and 71 in points t and t±; let L be its length, d and d\ the lengths of its pieces inside 
the discs D(t) and D(t\). If 7 and 71 are both dispersing then the field C is strictly preserved 
along this trajectory segment: diverging rays remain diverging. It follows that dispersing billiards 
are hyperbolic. If 7 is focusing and 71 is dispersing then the condition d < L implies the strict 
preservation of the field C. Indeed, if d < L then f — L < for all /o 6 [0, d\. Then the formula 
for the projective action implies that /1 < 0. The same condition works when 7 is dispersing and 
71 is focusing. Finally, if both curves 7 and 71 are focusing then the condition d + d\ < L implies 
the strict preservation of C. Indeed, in this case 

1 _ 2 1 
/1 di L-fo' 

If d + di < L then L - f > L - d > di, hence l//i > 1/di, or fi < d\. Also 2(L - / ) - di > 
2L — 2d — d\ > di > 0, hence /1 > 0. If d + di < L then the field of cones is preserved but not 
necessarily strictly preserved. 



fig. 126 
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Thus if the above conditions holds the corresponding billiard is hyperbolic. These conditions 
can be enforced in all cases but one by moving nonflat pieces of the boundary sufficiently far apart 
to make L big enough; the only remaining case is that of 7 = 71 being a focusing curve. Then 
the condition d + d\ < L, which should hold for each chord of the curve, imposes geometrical 
restrictions on it. We will describe such curves a little later; and now, as a matter of example, 
consider the billiard in a stadium. 



fig. 127 



Given a chord of a circle the relation d + d\ = L holds - see the figure. Therefore as long as 
a billiard trajectory reflects in one of the two stadium's semicircles the field of sectors is preserved 
but not strictly preserved. When a trajectory goes from one semicircle to another (possibly with 
intermediate reflections in the flat pieces) we have the inequality d + d\ < L, L being the length 
of the trajectory segment. In such a case the field of sectors is strictly preserved. Since almost 
every trajectory visits both semicircles, the field of sectors is eventually strictly preserved, and the 
billiard is hyperbolic. Notice that ergodicity does not follow from this argument; to establish it a 
more involved analysis is needed. 

Now we describe convex arcs satisfying Wojtkowski's condition. 

Lemma 3. The inequality d + d\ < L hold for every chord of a smooth convex arc 7 if and only 
if its radius of curvature r(t) is a strictly concave function of the length parameter t: 

Proof. Choose a coordinate system as shown. Let 4>{t) be the angle between the curve and 
the x-axis. Then 

dx , dy , . 1 d(f> 

— = cos (pit), — = sin (bit) and — — = — . 
dt Yy h dt Yy ' r(t) dt 

Also d = — r(to) sin (j)(t ), d\ = r(t\) sin (j>{t\). 
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We have 

f* 1 dx (t) , [ U x , f H dsin (f)(t) . . , 

L = / — — dt = / cos Mt) dt = / rit) dt 

k dt J t0 ^' J t0 dt 



f 1 dr 
= r(ti) sin 4>(t\) — r(to) sin (p(to) — / sin (p(t) — dt. 

Jto dt 

Hence 

, , f U ■ , / x dr , r* 1 dy(t) dr 

L-d-di = - sin d>(t) — dt = - / — 

Jto dt J tQ dt dt 



113 



= -«(*!) ^(ti) + v(to) Tt (to) + I y(t) W dt = l Q y(t) - 2 dt, 
because y(t\) = y(to) = 0. Since y(t) < for t G [<o,ii] the necessity follows. If 

d 2 r n 

at some point t then, choosing t and ii sufficiently close to t, one gets L — d — d\ < 0. Q.E.D. 
Notice that the condition 

d 2 r 

is open in C 4 topology. 

Here are some examples of the curves satisfying the condition 

d 2 r 

an arc of a circle; an arc of a logarithmic spiral r(t) = at + b; an arc of a cycloid r 2 (t) = —a 2 t 2 + b 2 ; 
an arc of an ellipse 

x 2 y 2 

— + ^2=1, a<b 

on which \x\ < a/y/2. 

Wojtkowski formulated the following principles for design of hyperbolic billiards: 
-any focusing component of the boundary should satisfy the inequality 

d 2 r 

-any focusing component should be sufficiently far away from any other component; 

-if two components meet at a vertex then the internal angle between them should be greater 
than 7r if both components are focusing, not less than tt if one is focusing and another dispersing, 
and greater than tt/2 if one is focusing and another neutral. 

(The last condition deals with reflections near a vertex.) 

Bunimovich billiards satisfy these conditions. Here are some other examples. 
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The first curve is the cardioid; the second is an elliptic stadium whose curves are great arcs of 
an ellipse. The third is a unit square with a hole in the shape of an astroid |x| 2 / 3 + |y| 2 ^ 3 = a 2 / 3 . 
If a < y/2/4 this billiard is hyperbolic. 

A more general class of convex curves that may be used to build hyperbolic billiards are 
absolutely focusing arcs. These are C 4 -smooth convex curves whose total rotation (i.e. the integral 
of curvature) does not exceed tt, and such that any incoming infinitesimal parallel beam focuses 
between each two consecutive reflections and focuses again after the last reflection (see [Bu 5, Don 
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2, Ch-M]). Bunimovich proved that a C 4 -small perturbation of an arc preserves its property to be 
absolutely focusing. Donnay showed that for any smooth convex arc there exists a > such that 
the arc is absolutely focusing for rays that make the angle with the arc less than a. This implies 
that any sufficiently short smooth strictly convex arc is absolutely focusing. 

fig. 130 

Arcs satisfying Wojtkowski's condition d + d\ < L are absolutely focusing. So are arcs for 
which Markarian's condition holds: d\(L + L{) < L\L (see [Mar 1]). A half-ellipse 

x 2 v 

is absolutely focusing if and only if a/6 < y/2 ([Don 2, Bu 5]). 

One may use absolutely focusing arcs instead of the ones satisfying Wojtkowski's condition, 
and construct hyperbolic billiards according to the above principles. An example is an elliptic 
stadium whose curves are small arcs of an ellipse; unlike the previous elliptic stadium here 

d 2 r 

fig. 131 

In conclusion we mention a possibility to design higher-dimensional hyperbolic billiards with 
focusing boundary components as discussed in [Bu 6]. However a serious difficulty arises in this 
case: linearly stable billiard trajectories cannot be destroyed by merely moving focusing pieces of 
the boundary far apart (see [Wo 4]). 

5.4 Miscellanea 

We saw in Section 3.6 that the dynamics of point-masses in the line reduces to a billiard in a 
polyhedral angle or a polyhedron. A more realistic physical model for gas deals with elastic balls, 
say n identical unit balls in 3-space. The configuration space S of this system is the subset of R 3n , 
corresponding to the positions of the ball's centers, in which the inequlities hold: 

(xi - Xj) 2 + (yi - yj) 2 + (zi - Zjf > 4; i^j, i,j = l,...,n. 

These inequalities say that the balls do not penetrate each other. Thus the space S is the com- 
plement of a union of cylinders. The dynamics of the system of balls is that of the billiard in S. 
Since a cylinder is a convex, but not strictly convex body, the billiard is semi-dispersing. 

The space S is kind of a higher-dimensional angle with faces convex inwards. Similarly to the 
case of a polyhedral angle a billiard trajectory eventually escapes from S (due to G. Galperin and 
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L. Vaserstein - see [G-C]). Hence the number of collisions of finitely many elastic balls in space 
is finite. Unlike the case of point-masses in the line this number is not proved to be uniformly 
bounded. 

A model of gas in a closed vessel is a collection of elastic balls inside a compact domain, 
say, a rectangular box. The famous Bolzmann's Hypothesis of statistical physics states that this 
system is ergodic on a constant energy level in the phase space. A slight variation of this model 
is a system of elastic balls in a torus. In a torus the total momentum and the center of mass 
are invariant; Bolzmann's Hypothesis concerns a level surface of these invariants in this case. 
Bolzmann's Hypothesis is not proved; the best known results are due to A. Kramli, N. Simanyi 
and D. Szasz ([K-S-S 3, 4]) who proved it for 3 and 4 balls in a torus of any dimension. 

Another popular model of statistical physics is the Lorentz gas which describes the motion 
of electrons in metals. This is the billiard in a domain in R n with a number of disjoint convex 
bodies (scatterers) removed. In particular, when the domain is the plane and the scatterers are 
periodically positioned identical discs one obtains the billiard in a torus with a circular hole. 

L. Bunimovich, C. Liverani, A. Pellegrinotti and Yu. Sukhov constructed a class of systems 
with an infinite number of elastic balls in which K-property holds on each component of a constant 
energy manifold ([B-L-P-S]). The following figure shows one such system. The "walls" are convex 
inward, and there are small scatterers inside the domain. The balls are too big to sqeeze through 
the narrow passages between the walls, but they can collide with each other and the scatterers. 
After a collision between two balls each ball first hits a wall or a scatterer, and only then may 
collide with another ball again. 

fig. 132 

Another result we want to mention here is an estimate from above for the number P n of n- 
periodic trajectories in a semi-dispersing billiard in any dimension, due to L. Stojanov ([Sto 2]). 
Let s be the number of smooth boundary components. 

Theorem 1. In a dispersing billiard 

P n < s(s - l) n (s - 2) for n > 3. 

In particular, 

lim sup < s — 1 . 

n— s-oo n 

This result complements the exponential estimate from below mentioned in Section 5.1. The 
proof is based on the observation that the length functional, whose critical points are periodic 
trajectories, is convex (in the plane it follows from a comparison of a boundary component with an 
appropriate ellipse - see the figure). It follows that in a dispersing billiard there cannot exist two 
distinct trajectories which consecutively reflect in the same boundary components, from which the 
theorem easily follows. In semi-despersing billiards trajectories that reflect in the same pieces of 
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the boundary appear in strips; each trajectory in a strip has the same length (compare to the case 
of polygons in Section 3.3). 

fig. 133 

We conclude this section with the following problem, discussed in [Ki]. A billiard table is 
bounded by the positive x-axis and a smooth curve y = f(x) > which is lower-asymptotic to 
the x-axis, that is liminf^oo /(x) = 0. A billiard ball is shot from a point on the y-axis in this 
billiard; is it possible that the ball will forever stay in the cusp of the table and never escape to 
the left half-plane? 

fig. 134 

Let V x be the set of unit vectors with the footpoints on the vertical segment through (x,0), 
equiped with its usual measure [i = sin a da dt from Section 1.2. We introduce the escape-set 
E C Vo which consists of phase points that never return to Vq. The first observation one makes 
concerns the case when the area below the curve is finite. Since the billiard flow is measure 
preserving it follows from Poincare's recurrence theorem that E is a nullset. 

If, in addition, the curve y = f(x) is convex then the escape-set is actually empty. Acute 
angles made by a billiard trajectory with the x-axis increase with each bounce in this case (/3 > a 
in figure 135). Hence the trajectory of a point in E must monotonically go to the right. Given 
v G E consider another vector w which makes a smaller angle with the horizontal direction, and 
whose footpoint is to the right of the footpoint of v. After a reflection in the upper and then in 
the lower boundaries the ball w is still to the right of v and still makes a smaller angle with the 
x-axis. Iterating we see that w escapes as well. Hence E cannot be a nullset unless it is void. 

fig. 135 

Finally let the table have infinite area. We claim that E is still a nullset. Assume that it has 
a positive measure. By dropping to a positive measure subset we may assume that for all v G E 

lim sup x-coordinate of 7t(u) = +oo, 

t — >oo 

where -ft is the billiard flow. Indeed, otherwise there will be a subset U of positive measure in E 
such that the trajectories of its points stay in a bounded part of the billiard table to the left of 
some vertical line; as before, one may apply Poincare's recurrence theorem to U. Thus for any 
x > each point v G E will eventually reach V x , i.e. 7t(t>) G V x for some t > 0. Let T : E — > V x 
be the corresponding billiard transformation which sends a vector to the first point of V x on its 
forward trajectory. Since T is measure preserving jjl (V x ) > /i (E). However the curve y = f(x) is 
lower- asymptotic, hence \i (V x ) can be made arbitrarily small by choosing x big enough. This is a 
contradiction. As J. King puts it, one cannot squeeze a gallon into a bottle with a pint-sized neck. 
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fig. 136 



It follows that in a billiard, bounded by a curve lower-asymptotic in both directions, almost 
every trajectory which starts at the y-axis will visit it again infinitely many times. Such billiards, 
which are models for the motion of a charge in an electro-magnetic field of a special configuration, 
were studied by A. Leontovich in early 60-s. 
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